Paper 2509.21268
MMR1: Enhancing Multimodal Reasoning with Variance-Aware Sampling and Open Resources
- Published
- Sep 2025
- Research lab
- Independent
- Citations
- 35
- GitHub
- 217 stars
01 In brief
Summary
This paper introduces MMR1, a framework for enhancing multimodal reasoning in large models.
It addresses two key limitations: the lack of open, large-scale, high-quality long chain-of-thought (CoT) data, and the instability of reinforcement learning (RL) algorithms, specifically gradient vanishing in Group Relative Policy Optimization (GRPO) when reward variance is low.
The authors propose Variance-Aware Sampling (VAS), a data selection strategy using a Variance Promotion Score (VPS) that combines Outcome Variance Score (OVS) and Trajectory Diversity Score (TDS) to promote reward variance and stabilize training.
They release curated resources: ~1.6M long CoT cold-start data and ~15k RL QA pairs, along with a reproducible codebase and open-source models at multiple scales.
Experiments on mathematical reasoning benchmarks (MathVerse, MathVista, MathVision, LogicVista, ChartQA) show that MMR1-7B achieves state-of-the-art average performance (58.4) among reasoning-oriented models, and MMR1-3B is competitive.
Ablation studies confirm the complementary benefits of OVS and TDS.
Theoretically, they prove that reward variance lower-bounds expected policy gradient magnitude, with VAS as a practical mechanism.
Limitations include residual training instabilities and computational overhead for VPS computation.
02 From the paper
Abstract
Large multimodal reasoning models have achieved rapid progress, but their advancement is constrained by two major limitations: the absence of open, large-scale, high-quality long chain-of-thought (CoT) data, and the instability of reinforcement learning (RL) algorithms in post-training. Group Relative Policy Optimization (GRPO), the standard framework for RL fine-tuning, is prone to gradient vanishing when reward variance is low, which weakens optimization signals and impairs convergence. This work makes three contributions: (1) We propose Variance-Aware Sampling (VAS), a data selection strategy guided by Variance Promotion Score (VPS) that combines outcome variance and trajectory diversity to promote reward variance and stabilize policy optimization. (2) We release large-scale, carefully curated resources containing ~1.6M long CoT cold-start data and ~15k RL QA pairs, designed to ensure quality, difficulty, and diversity, along with a fully reproducible end-to-end training codebase. (3) We open-source a family of multimodal reasoning models in multiple scales, establishing standardized baselines for the community. Experiments across mathematical reasoning benchmarks demonstrate the effectiveness of both the curated data and the proposed VAS. Comprehensive ablation studies and analyses provide further insight into the contributions of each component. In addition, we theoretically establish that reward variance lower-bounds the expected policy gradient magnitude, with VAS serving as a practical mechanism to realize this guarantee. Our code, data, and checkpoints are available at https://github.com/LengSicong/MMR1.