Paper 2510.08540
MM-HELIX: Boosting Multimodal Long-Chain Reflective Reasoning with Holistic Platform and Adaptive Hybrid Policy Optimization
- Published
- Oct 2025
- Research lab
- Independent
- Citations
- 3
- GitHub
- 69 stars
01 In brief
Summary
The paper introduces MM-HELIX, a benchmark of 1,260 samples across 42 multimodal tasks (algorithms, graphs, puzzles, games) with five difficulty levels, designed to evaluate long-chain reflective reasoning in MLLMs.
Evaluation of 23 models shows significant deficits; even GPT-5 achieves only 58.1% accuracy, and open-source models lag far behind.
To improve this capability, the authors propose the Step-Elicited Response Generation (SERG) pipeline, which combines rule-based CoT scaffolds with LLM enhancement to create MM-HELIX-100K, a dataset of 100k high-quality reflective reasoning traces.
They also introduce Adaptive Hybrid Policy Optimization (AHPO), a training method that dynamically combines off-policy expert supervision with on-policy exploration, using a reward-based gating mechanism to activate expert guidance when rewards are sparse and deactivate it when the model becomes proficient.
Applied to Qwen2.5-VL-7B, AHPO achieves a +18.6% accuracy improvement on MM-HELIX and a +5.7% average gain on general math and logic tasks, demonstrating that reflective reasoning can be learned and generalized.
The work addresses the limitations of standard SFT and RL methods, which suffer from catastrophic forgetting and sparse rewards, respectively.
02 From the paper
Abstract
While current Multimodal Large Language Models (MLLMs) have demonstrated proficiency in reasoning tasks such as mathematics and logic, their capacity for long-chain reflective reasoning, a prerequisite for solving complex real-world problems, remains largely underexplored. In this work, we first conduct an extensive empirical investigation to evaluate this capability. Leveraging a carefully designed data synthesis engine, we construct MM-HELIX, a multimodal benchmark consisting 1,260 samples of 42 challenging synthetic tasks that require iterative thinking and backtracking. Empirical results on this benchmark reveal that existing MLLMs exhibit significant performance deficits in long-chain reflective reasoning. To address this limitation, we generate post-training data and further explore learning paradigms for exploiting such data. We first develop the Step-Elicited Response Generation pipeline to create MM-HELIX-100K, a large-scale dataset of 100k high-quality, reflective reasoning traces for instruction-tuning stage. Given that standard Reinforcement Learning fails on complex tasks due to sparse reward signals and catastrophic forgetting after Supervised Fine-Tuning, we propose Adaptive Hybrid Policy Optimization (AHPO), a novel training strategy that dynamically unifies offline supervision and online optimization into a single stage. This strategy enables the model to learn from expert data when rewards are sparse and conduct independent exploration once proficient. When applied to the Qwen2.5-VL-7B baseline, our method achieves a +18.6\% accuracy improvement on MM-HELIX benchmark and demonstrates strong generalization with a +5.7\% average performance gain on general mathematic and logic tasks. Our work demonstrate that reflective reasoning in MLLMs can be effectively learned and generalized, paving the way for developing more capable MLLMs.