Paper 2510.18855
Every Step Evolves: Scaling Reinforcement Learning for Trillion-Scale Thinking Model
- Published
- Oct 2025
- Research lab
- Independent
- Citations
- 32
- GitHub
- Not linked
01 In brief
Summary
Ring-1T is the first open-source, state-of-the-art thinking model with 1 trillion total parameters and about 50 billion activated per token, built on the Ling 2.0 architecture.
Training at this scale posed challenges like train-inference misalignment, rollout inefficiencies, and RL system bottlenecks.
The authors introduced three innovations: IcePop, which stabilizes RL training by masking and clipping token-level probability discrepancies; C3PO++, which dynamically partitions long rollouts under a token budget to improve resource utilization; and ASystem, a high-performance RL framework with components like Hybrid Runtime, AMem, AState, and ASandbox.
The model was trained via long-CoT SFT, reasoning RL, and general RL stages.
Ring-1T achieves strong results: 93.4 on AIME-2025, 86.72 on HMMT-2025, 2088 on CodeForces, 55.94 on ARC-AGI-1, and a silver-medal-level performance on IMO-2025.
It outperforms existing open-source models on many benchmarks and is competitive with closed-weights models.
The full model is released to the community, marking a milestone in democratizing large-scale reasoning intelligence.
02 From the paper
Abstract
We present Ring-1T, the first open-source, state-of-the-art thinking model with a trillion-scale parameter. It features 1 trillion total parameters and activates approximately 50 billion per token. Training such models at a trillion-parameter scale introduces unprecedented challenges, including train-inference misalignment, inefficiencies in rollout processing, and bottlenecks in the RL system. To address these, we pioneer three interconnected innovations: (1) IcePop stabilizes RL training via token-level discrepancy masking and clipping, resolving instability from training-inference mismatches; (2) C3PO++ improves resource utilization for long rollouts under a token budget by dynamically partitioning them, thereby obtaining high time efficiency; and (3) ASystem, a high-performance RL framework designed to overcome the systemic bottlenecks that impede trillion-parameter model training. Ring-1T delivers breakthrough results across critical benchmarks: 93.4 on AIME-2025, 86.72 on HMMT-2025, 2088 on CodeForces, and 55.94 on ARC-AGI-1. Notably, it attains a silver medal-level result on the IMO-2025, underscoring its exceptional reasoning capabilities. By releasing the complete 1T parameter MoE model to the community, we provide the research community with direct access to cutting-edge reasoning capabilities. This contribution marks a significant milestone in democratizing large-scale reasoning intelligence and establishes a new baseline for open-source model performance.