Paper 2602.04145
Training Data Efficiency in Multimodal Process Reward Models
- Published
- Feb 2026
- Research lab
- Independent
- Citations
- 1
- GitHub
- 7 stars
01 In brief
Summary
The paper studies data efficiency for training Multimodal Process Reward Models (MPRMs), which provide step-level supervision for visual reasoning in Multimodal Large Language Models.
Training MPRMs typically requires large-scale Monte Carlo (MC)-annotated corpora, which is computationally expensive.
The authors find that random subsampling of training data quickly saturates performance, indicating redundancy.
They formalize a theoretical framework showing that informative gradient updates depend on label mixture (positive/negative steps) and label reliability (average MC scores of positive steps).
Based on this, they propose the Balanced-Information Score (BIS), which ranks rollouts by mixture and reliability using existing MC signals without extra cost.
Experiments on VisualProcessBench with InternVL2.5-8B and Qwen2.5-VL-7B show that BIS-selected subsets match or surpass full-data performance at small fractions, e.g., 10% of data achieves full-data performance, improving over random subsampling by 4.1% relative.
The code is released as Balanced-Info-MPRM.
02 From the paper
Abstract
Multimodal Process Reward Models (MPRMs) are central to step-level supervision for visual reasoning in MLLMs. Training MPRMs typically requires large-scale Monte Carlo (MC)-annotated corpora, incurring substantial training cost. This paper studies the data efficiency for MPRM training. Our preliminary experiments reveal that MPRM training quickly saturates under random subsampling of the training data, indicating substantial redundancy within existing MC-annotated corpora. To explain this, we formalize a theoretical framework and reveal that informative gradient updates depend on two factors: label mixtures of positive/negative steps and label reliability (average MC scores of positive steps). Guided by these insights, we propose the Balanced-Information Score (BIS), which prioritizes both mixture and reliability based on existing MC signals at the rollout level, without incurring any additional cost. Across two backbones (InternVL2.5-8B and Qwen2.5-VL-7B) on VisualProcessBench, BIS-selected subsets consistently match and even surpass the full-data performance at small fractions. Notably, the BIS subset reaches full-data performance using only 10% of the training data, improving over random subsampling by a relative 4.1%.