Paper 2607.24653
Kimi K3: Open Frontier Intelligence
- Published
- Jul 2026
- Research lab
- Moonshot AI
- Citations
- 2
- GitHub
- 8.1K stars
01 In brief
Summary
Kimi K3 is a 2.8-trillion-parameter Mixture-of-Experts model with 104 billion activated parameters, native vision, and a 1-million-token context window.
It uses Kimi Delta Attention, Attention Residuals, and Stable LatentMoE (16 of 896 routed experts active per token), achieving about 2.5× scaling efficiency over Kimi K2.
Post-training includes reinforcement learning across general, agentic, and coding domains with multiple reasoning-effort levels, consolidated via multi-teacher on-policy distillation.
Infrastructure innovations include KDA co-design, MoonEP for balanced expert-parallel training, and million-token agentic RL with resumable sandboxes.
Evaluations show frontier-level performance, trailing only Claude Fable 5 and GPT-5.6 Sol, while outperforming other open and proprietary models.
The full model weights are released to facilitate research and adoption.
Key results include 67.5% on DeepSWE, 91.2% on BrowseComp, 93.5% on GPQA Diamond, and 41.0% on ZeroBench-main with tools.
The model also demonstrates strong cost efficiency, achieving near-top scores at a fraction of the cost of leading proprietary models.
Case studies show capabilities in GPU kernel optimization, compiler development, chip design, and research coding.
The model's cyber capabilities are strongest in vulnerability discovery and user-space exploitation, with a clear gap to human experts on hardened targets.
Third-party evaluations rank it fourth on Artificial Analysis Intelligence Index and second on Vals AI Index.
The model is the first open 3T-class model, establishing a new open frontier.
02 From the paper
Abstract
We introduce Kimi K3, a 2.8T parameter Mixture-of-Experts model with 104 billion activated parameters, native vision capabilities, and a 1-million-token context window. Kimi K3 is built on Kimi Delta Attention and Attention Residuals, which improve information flow across sequence length and model depth. Together with Stable LatentMoE, which effectively activates 16 of 896 routed experts per token, and refined training and data recipes, these advances yield an approximately 2.5x improvement in overall scaling efficiency over Kimi K2. Post-training highlights reinforcement learning across general, agentic, and coding domains and multiple reasoning-effort levels, enabling compositional generalization and robust long-horizon execution. At 2.8T scale, Kimi K3 is supported by infrastructure advances in multiple areas: algorithm-system co-design for KDA, perfectly balanced expert-parallel training with efficient memory management, million-token agentic RL with persistent rollout and sandbox states, and deployment innovations. Extensive evaluations show that Kimi K3 achieves frontier-level performance across long-horizon coding, agentic, knowledge, reasoning, and vision tasks. While its overall performance still trails the most powerful proprietary models, namely Claude Fable 5 and GPT-5.6 Sol, Kimi K3 consistently outperforms other open and proprietary models evaluated in our suite. We release the full Kimi K3 model weights to facilitate future research and accelerate the broader deployment and adoption of frontier intelligence.