Paper 2602.10604
Step 3.5 Flash: Open Frontier-Level Intelligence with 11B Active Parameters
- Published
- Feb 2026
- Research lab
- Independent
- Citations
- 25
- GitHub
- 2.1K stars
01 In brief
Summary
Step 3.5 Flash is a sparse Mixture-of-Experts (MoE) language model with 196B total parameters and 11B active parameters, designed to deliver frontier-level agentic intelligence with high computational efficiency.
Its architecture uses a 3:1 interleaved Sliding Window/Full Attention layout, head-wise gated attention, and Multi-Token Prediction (MTP-3) to minimize latency and cost for long-context agentic interactions.
The model was trained on 17.6T tokens with a focus on stability, addressing issues like Muon optimizer numerical sensitivity, expert collapse, and activation blow-ups.
Post-training introduces a unified framework with domain-specific expert models, self-distillation, and a novel MIS-PO reinforcement learning algorithm for stable large-scale off-policy training.
Step 3.5 Flash achieves strong results on reasoning and agentic benchmarks, including 85.4% on IMO-AnswerBench, 86.4% on LiveCodeBench-v6, 88.2% on τ²-Bench, 69.0% on BrowseComp (with context management), and 51.0% on Terminal-Bench 2.0, performing on par with frontier models like GPT-5.2 xHigh and Gemini 3.0 Pro.
The model is designed for real-world industrial deployment, sustaining ~170 tokens/s on Hopper GPUs, and demonstrates strong tool-use and long-horizon decision-making capabilities.
02 From the paper
Abstract
We introduce Step 3.5 Flash, a sparse Mixture-of-Experts (MoE) model that bridges frontier-level agentic intelligence and computational efficiency. We focus on what matters most when building agents: sharp reasoning and fast, reliable execution. Step 3.5 Flash pairs a 196B-parameter foundation with 11B active parameters for efficient inference. It is optimized with interleaved 3:1 sliding-window/full attention and Multi-Token Prediction (MTP-3) to reduce the latency and cost of multi-round agentic interactions. To reach frontier-level intelligence, we design a scalable reinforcement learning framework that combines verifiable signals with preference feedback, while remaining stable under large-scale off-policy training, enabling consistent self-improvement across mathematics, code, and tool use. Step 3.5 Flash demonstrates strong performance across agent, coding, and math tasks, achieving 85.4% on IMO-AnswerBench, 86.4% on LiveCodeBench-v6 (2024.08-2025.05), 88.2% on tau2-Bench, 69.0% on BrowseComp (with context management), and 51.0% on Terminal-Bench 2.0, comparable to frontier models such as GPT-5.2 xHigh and Gemini 3.0 Pro. By redefining the efficiency frontier, Step 3.5 Flash provides a high-density foundation for deploying sophisticated agents in real-world industrial environments.