StepFun Open-Sources Step 5 Preview Weights for Long-Horizon Agent Tasks

On October 15, StepFun announced the open release of its Step 5 Preview model weights, including BF16 and FP8 versions, along with a technical blog and open-source development tools. The weights are now available on Hugging Face and ModelScope.

The model is designed for long-horizon agent tasks and adopts a sparse Mixture-of-Experts architecture with 600B total parameters and approximately 27B activated parameters per token. It supports a 1 million-token context window and natively handles text and vision inputs. According to StepFun, the model recently ranked second globally on OpenRouter's weekly and daily call volume charts.

Developers can deploy an OpenAI-compatible API service using vLLM or SGLang, or run offline inference with Transformers, integrating the model into their own data, tools, and systems to build agents, applications, workflows, and services. The company also said the release includes adaptation support for multiple chip platforms and inference frameworks.

The accompanying technical blog describes co-optimizations in architecture and training, including a 92-layer narrow-and-deep architecture for deeper implicit multi-hop reasoning paths, Sparse GQA with block-level indexing and sparse computation to reduce indexing and top-k selection overhead in million-token contexts to one-eighth, FP32 retention in critical states and parameters with fine-grained gradient clipping for training stability, a million-level verifiable environment covering software engineering, scientific reasoning, ML research and knowledge work, and a long-horizon RL method that incorporates context compression into end-to-end policy training.