Paper 2508.21113
R-4B: Incentivizing General-Purpose Auto-Thinking Capability in MLLMs via Bi-Mode Annealing and Reinforce Learning
- Published
- Aug 2025
- Research lab
- Independent
- Citations
- 16
- GitHub
- 141 stars
01 In brief
Summary
The paper introduces R-4B, a multimodal large language model (MLLM) with auto-thinking capability, which adaptively decides whether to engage in step-by-step reasoning based on problem complexity.
R-4B is trained in two stages: bi-mode annealing, which uses a curated dataset of reasoning and non-reasoning examples to equip the model with both thinking and non-thinking modes, and Bi-mode Policy Optimization (BPO), a reinforcement learning method that forces the model to generate responses in both modes for each query, using a simple rule-based reward.
This approach avoids complex reward engineering and hyperparameter sensitivity.
Evaluations on 25 benchmarks show R-4B-RL outperforms Qwen2.5-VL-7B on most tasks and matches the larger Kimi-VL-A3B-Thinking-2506 (16B) on reasoning-intensive benchmarks, while reducing token consumption on simple tasks.
The model achieves state-of-the-art results on several benchmarks, including MMMU (68.1%), CharXiv-RQ (56.8%), and MathVerse-vision (64.9%).
The paper also analyzes the thinking triggering rate during training, showing BPO effectively learns to activate thinking for complex tasks and avoid it for simple ones, mitigating 'thinking atrophy'.
02 From the paper
Abstract
Multimodal Large Language Models (MLLMs) equipped with step-by-step thinking capabilities have demonstrated remarkable performance on complex reasoning problems. However, this thinking process is redundant for simple problems solvable without complex reasoning. To address this inefficiency, we propose R-4B, an auto-thinking MLLM, which can adaptively decide when to think based on problem complexity. The central idea of R-4B is to empower the model with both thinking and non-thinking capabilities using bi-mode annealing, and apply Bi-mode Policy Optimization (BPO) to improve the model's accuracy in determining whether to activate the thinking process. Specifically, we first train the model on a carefully curated dataset spanning various topics, which contains samples from both thinking and non-thinking modes. Then it undergoes a second phase of training under an improved GRPO framework, where the policy model is forced to generate responses from both modes for each input query. Experimental results show that R-4B achieves state-of-the-art performance across 25 challenging benchmarks. It outperforms Qwen2.5-VL-7B in most tasks and achieves performance comparable to larger models such as Kimi-VL-A3B-Thinking-2506 (16B) on reasoning-intensive benchmarks with lower computational cost.