Paper 2601.09668
STEP3-VL-10B Technical Report
- Published
- Jan 2026
- Research lab
- Independent
- Citations
- 26
- GitHub
- 412 stars
01 In brief
Summary
Step3-VL-10B is a 10B-parameter open-source multimodal foundation model that rivals or surpasses models 10-20x larger, such as GLM-4.6V-106B and Qwen3-VL-235B, and proprietary systems like Gemini 2.5 Pro.
It achieves 92.2% on MMBench, 80.11% on MMMU, 94.43% on AIME2025, and 75.95% on MathVision.
The model uses a unified pre-training strategy on 1.2T multimodal tokens, integrating a language-aligned Perception Encoder with a Qwen3-8B decoder.
Post-training includes over 1k iterations of reinforcement learning (RLVR and RLHF) and implements Parallel Coordinated Reasoning (PaCoRe) to scale test-time compute, enabling parallel exploration and synthesis of visual hypotheses.
The model excels in perception, reasoning, OCR, GUI grounding, and spatial understanding, and maintains strong text-centric performance.
Ablations show the Perception Encoder outperforms DINOv3, and AdamW is preferred over Muon due to initialization issues.
RL dynamics reveal distinct length patterns: reasoning tasks lengthen, while perception tasks shorten due to entropy reduction.
PaCoRe addresses the 'missing trace' hypothesis by externalizing implicit visual processes, leading to significant gains in perception and reasoning benchmarks.
The model is released to the community as an efficient baseline for multimodal intelligence.
02 From the paper
Abstract
We present STEP3-VL-10B, a lightweight open-source foundation model designed to redefine the trade-off between compact efficiency and frontier-level multimodal intelligence. STEP3-VL-10B is realized through two strategic shifts: first, a unified, fully unfrozen pre-training strategy on 1.2T multimodal tokens that integrates a language-aligned Perception Encoder with a Qwen3-8B decoder to establish intrinsic vision-language synergy; and second, a scaled post-training pipeline featuring over 1k iterations of reinforcement learning. Crucially, we implement Parallel Coordinated Reasoning (PaCoRe) to scale test-time compute, allocating resources to scalable perceptual reasoning that explores and synthesizes diverse visual hypotheses. Consequently, despite its compact 10B footprint, STEP3-VL-10B rivals or surpasses models 10$\times$-20$\times$ larger (e.g., GLM-4.6V-106B, Qwen3-VL-235B) and top-tier proprietary flagships like Gemini 2.5 Pro and Seed-1.5-VL. Delivering best-in-class performance, it records 92.2% on MMBench and 80.11% on MMMU, while excelling in complex reasoning with 94.43% on AIME2025 and 75.95% on MathVision. We release the full model suite to provide the community with a powerful, efficient, and reproducible baseline.