DeepSeek Open-Sources DeepInfer, Its Internal Inference Framework

DeepSeek announced on October 20 that it has fully open-sourced its internal inference framework, DeepInfer, opening up its large-model inference infrastructure to help developers and enterprises reduce compute costs and energy consumption in large-scale AI services.

According to DeepSeek, DeepInfer integrates high-performance operators and an optimized inference engine, supporting distributed inference, speculative decoding, and structured output. It is compatible with both NVIDIA and Huawei Ascend platforms. Based on this framework, DeepSeek has delivered high-throughput, low-latency inference services for DeepSeek V4.1 Flash.

DeepSeek also released deepseek-recipe, a production-grade token encoding and decoding component, to ensure the community gets an API experience consistent with its official deployment. Future inference code and optimizations for new DeepSeek models will be continuously updated in DeepInfer.

The company said it hopes that by opening up models, operator libraries, and inference frameworks, developers can improve inference efficiency, reduce deployment costs, and make AI services accessible to a wider user base at lower cost.