GitHub Copilot to support on-device AI inference by end of October

Microsoft announced at an event in San Francisco on October 7 that GitHub Copilot will support on-device AI model inference before the end of October 2026. Developers will be able to switch automatically or manually between cloud models and device-side models, and set preferences in GitHub Copilot CLI, the Copilot app, and Visual Studio Code.

GitHub Copilot currently relies on cloud-hosted models, with a coordinator routing requests based on performance, cost, and accuracy. Microsoft plans to extend this mechanism so Copilot can choose between cloud and local AI models while coordinating inference tasks in the background. Users can specify a provider, model, or endpoint, using Windows ML to select MAI Code 1.1 Flash or connecting to an OpenAI-compatible local endpoint.

Microsoft also introduced MAI Code 1.1 Flash, a mixture-of-experts model with 137 billion total parameters and 6.8 billion active parameters. It uses quantization and speculative decoding to optimize response speed and memory usage. On the Surface Laptop Ultra, decode throughput ranged from 40 to 63 tokens per second for prompt lengths between 2K and 256K tokens.