The Year in AI Papers
The year/Independent research

Paper 2606.30616

Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agent

Lei Bai, Zongsheng Cao, Yang Chen, Zhiyao Cui, Shangheng Du, Yue Fan, Shiyang Feng, Zijie Guo, Haonan He, Liang He, Xiaohan He, Shuyue Hu, et al.

Published
Jun 2026
Research lab
Independent
Citations
3
GitHub
529 stars

01 In brief

Summary

Agents-A1 is a 35B Mixture-of-Experts agentic model that achieves trillion-parameter-level performance by scaling the agent horizon rather than parameters.

The authors built a long-horizon knowledge-action infrastructure that connects external knowledge, actions, observations, and verifier outcomes, producing agentic trajectories averaging 45K tokens.

Training uses a three-stage recipe: full-domain supervised fine-tuning, domain-level teacher training, and multi-teacher domain-routed on-policy distillation with salient vocabulary alignment to unify six heterogeneous domains.

Agents-A1 outperforms 1T-parameter models (Kimi-K2.6, DeepSeek-V4-pro) on SEAL-0 (56.4), IFBench (80.6), HiPhO (46.4), FrontierScience-Olympiad (79.0), and MolBench-Bind (56.8), and is competitive on SciCode (44.3), HLE (47.6), and BrowseComp (75.5).

The model also demonstrates long-horizon capabilities in a 12-hour machine learning optimization run (improving validation AUC from 0.58 to 0.9935) and in Earth science analysis.

The work provides a practical path for scaling the horizon using a 35B agent to match or exceed 1T models on long-horizon tasks.

02 From the paper

Abstract

We introduce Agents-A1, a 35B Mixture-of-Experts Agentic Model that reaches trillion-parameter-level performance by scaling the agent horizon. We investigate agent-horizon scaling from two perspectives: scaling long-horizon trajectories and scaling heterogeneous agent abilities. To support this goal, we build a long-horizon knowledge-action infrastructure that connects external knowledge, actions, observations, and verifier outcomes, producing agentic trajectories with an average length of 45K tokens. Based on this, we train Agents-A1 with a three-stage recipe. First, we perform full-domain supervised fine-tuning to align the base model with broad agentic behaviors. Second, we train domain-level teacher models to capture specialized expertise in each domain. Third, we propose a multi-teacher domain-routed on-policy distillation with salient vocabulary alignment to improve knowledge transfer efficiency across different domains, unifying six heterogeneous domains into one deployable student model. Agents-A1 achieves strong and broad performance for long-horizon agent benchmarks. Compared with 1T-parameter model such as Kimi-K2.6 and DeepSeek-V4-pro, Agents-A1 achieves leading results on SEAL-0 (56.4), IFBench (80.6), HiPhO (46.4), FrontierScience-Olympiad (79.0), and MolBench-Bind (56.8), and remains highly competitive on SciCode (44.3), HLE (47.6) and BrowseComp (75.5). We hope this work provides the community with a practical path for scaling the horizon using a 35B agent that can reach or match the performance of 1T models on long-horizon tasks.