Paper 2512.24618
Youtu-LLM: Unlocking the Native Agentic Potential for Lightweight Large Language Models
- Published
- Dec 2025
- Research lab
- Independent
- Citations
- 5
- GitHub
- 593 stars
01 In brief
Summary
Youtu-LLM is a 1.96B-parameter language model pre-trained from scratch to achieve native agentic intelligence, balancing efficiency with strong reasoning and planning.
It uses a dense Multi-Latent Attention (MLA) architecture with a 128k context window and a STEM-oriented tokenizer.
Training follows a 'Commonsense-STEM-Agent' curriculum over 10.84T tokens, including 200B tokens of agentic trajectory data (Agentic-CoT, math, code, deep research, tool-use).
Post-training includes two-stage SFT and reinforcement learning with FP16 precision and consistent sampling for stability.
Evaluations show Youtu-LLM outperforms similar-sized models and rivals larger ones (e.g., Qwen3-4B) on general and agentic benchmarks, with significant gains on SWE-Bench-Verified (17.7% vs 12.4% without agentic mid-training) and GAIA (33.9%).
The work demonstrates that lightweight models can acquire strong agentic capabilities through pre-training, not just post-hoc augmentation.
02 From the paper
Abstract
We introduce Youtu-LLM, a lightweight yet powerful language model that harmonizes high computational efficiency with native agentic intelligence. Unlike typical small models that rely on distillation, Youtu-LLM (1.96B) is pre-trained from scratch to systematically cultivate reasoning and planning capabilities. The key technical advancements are as follows: (1) Compact Architecture with Long-Context Support: Built on a dense Multi-Latent Attention (MLA) architecture with a novel STEM-oriented vocabulary, Youtu-LLM supports a 128k context window. This design enables robust long-context reasoning and state tracking within a minimal memory footprint, making it ideal for long-horizon agent and reasoning tasks. (2) Principled "Commonsense-STEM-Agent" Curriculum: We curated a massive corpus of approximately 11T tokens and implemented a multi-stage training strategy. By progressively shifting the pre-training data distribution from general commonsense to complex STEM and agentic tasks, we ensure the model acquires deep cognitive abilities rather than superficial alignment. (3) Scalable Agentic Mid-training: Specifically for the agentic mid-training, we employ diverse data construction schemes to synthesize rich and varied trajectories across math, coding, and tool-use domains. This high-quality data enables the model to internalize planning and reflection behaviors effectively. Extensive evaluations show that Youtu-LLM sets a new state-of-the-art for sub-2B LLMs. On general benchmarks, it achieves competitive performance against larger models, while on agent-specific tasks, it significantly surpasses existing SOTA baselines, demonstrating that lightweight models can possess strong intrinsic agentic capabilities.