Paper 2606.16140
VibeThinker-3B: Exploring the Frontier of Verifiable Reasoning in Small Language Models
- Published
- Jun 2026
- Research lab
- Independent
- Citations
- 1
- GitHub
- 1.5K stars
01 In brief
Summary
VibeThinker-3B is a 3B-parameter dense model from Sina Weibo Inc.
that achieves frontier-level performance on verifiable reasoning tasks, matching or exceeding much larger models.
Built on Qwen2.5-Coder-3B, it uses a post-training pipeline with curriculum SFT, multi-domain RL (math, code, STEM), Long2Short Math RL for efficiency, offline self-distillation, and Instruct RL.
Key results include 94.3 on AIME26 (97.1 with Claim-Level Reliability Assessment, CLR), 80.2 Pass@1 on LiveCodeBench v6, and 96.1% acceptance on recent LeetCode contests.
It also scores 93.4 on IFEval, showing preserved instruction control.
The authors propose the Parametric Compression-Coverage Hypothesis: verifiable reasoning is compressible into a compact reasoning core, while open-domain knowledge requires broad parameter coverage.
This suggests small models can complement large-scale generalists in reasoning-dense domains, though gaps remain on knowledge-heavy benchmarks like GPQA-Diamond (70.2, 72.9 with CLR).
The model's performance rivals DeepSeek V3.2, GLM-5, and Gemini 3 Pro on reasoning tasks, demonstrating that parameter scale is not the sole determinant of reasoning capability.
02 From the paper
Abstract
This technical report introduces VibeThinker-3B, a compact dense model with 3B parameters developed to investigate how far verifiable reasoning can be pushed within a strictly small-model regime. Building upon the Spectrum-to-Signal post-training paradigm, we systematically enhance the model through an optimized pipeline that includes curriculum-based supervised fine-tuning, multi-domain reinforcement learning, and offline self-distillation. Experimental evaluations demonstrate that VibeThinker-3B achieves frontier-level performance on highly demanding verifiable tasks. Specifically, it attains a score of 94.3 on AIME26 (improving to 97.1 with claim-level test-time scaling), an 80.2 Pass@1 on LiveCodeBench v6, and exhibits strong out-of-distribution generalization with a 96.1\% acceptance rate on recent unseen LeetCode contests. This effectively places it in the performance band of first-tier reasoning systems, matching or exceeding flagship models that are orders of magnitude larger, such as DeepSeek V3.2, GLM-5, and Gemini 3 Pro. Furthermore, a score of 93.4 on IFEval confirms that this extreme reasoning enhancement does not compromise strict instruction controllability. Extending our previous 1.5B work, these findings motivate the Parametric Compression-Coverage Hypothesis, which views verifiable reasoning as compressible into compact reasoning cores, while open-domain knowledge and general-purpose competence require broad parameter coverage over facts, concepts, and long-tail scenarios. This perspective suggests that compact models are not merely deployment-efficient substitutes, but a complementary path toward frontier-level performance in parameter-dense capability regimes.