Paper 2602.22859
From Blind Spots to Gains: Diagnostic-Driven Iterative Training for Large Multimodal Models
- Published
- Feb 2026
- Research lab
- Independent
- Citations
- 0
- GitHub
- 31 stars
01 In brief
Summary
The paper introduces Diagnostic-driven Progressive Evolution (DPE), a closed-loop training framework for Large Multimodal Models (LMMs) that addresses limitations of static data and heuristic self-evolution methods.
DPE iteratively performs diagnosis, targeted data generation, and reinforcement learning.
A diagnostic agent identifies capability blind spots across 12 categories, producing a report that guides a multi-agent system (Planner, Image Selector, Question Generator, Validation) to generate diverse, weakness-focused training samples using external image pools and editing tools.
Experiments on Qwen2.5-VL-7B-Instruct and Qwen3-VL-8B-Instruct show DPE achieves stable, continual gains across 11 benchmarks, outperforming VisPlay and matching or exceeding larger models (e.g., Qwen2.5-VL-72B) with only ~3,000 generated samples.
Ablations confirm the diagnostic module and image tools are crucial for stability and long-tail coverage.
DPE also improves text and image diversity and question quality compared to VisPlay.
The code, models, and data are publicly available at https://github.com/hongruijia/DPE.
The work is from Peking University and Shandong University, with authors Hongrui Jia, Chaoya Jiang, Yongrui Heng, Shikun Zhang, and Wei Ye.
The framework is evaluated on benchmarks including MMMU, MathVision, CharXiv, and HallusionBench, achieving an average score of 64.39 on an 8B backbone, surpassing GPT-4o (56.1) and Qwen2.5-VL-72B (61.9).
DPE's diagnostic mechanism dynamically adjusts data mixture ratios, concentrating resources on weak categories, leading to improvements like CharXiv accuracy rising from 36.8 to 40.91 over three iterations.
The method uses GRPO for optimization and retains only moderately difficult samples for efficient learning.
The paper also provides a theoretical analysis linking learning efficiency to reward variance, supporting the difficulty-aware filtering strategy…
02 From the paper
Abstract
As Large Multimodal Models (LMMs) scale up and reinforcement learning (RL) methods mature, LMMs have made notable progress in complex reasoning and decision making. Yet training still relies on static data and fixed recipes, making it difficult to diagnose capability blind spots or provide dynamic, targeted reinforcement. Motivated by findings that test driven error exposure and feedback based correction outperform repetitive practice, we propose Diagnostic-driven Progressive Evolution (DPE), a spiral loop where diagnosis steers data generation and reinforcement, and each iteration re-diagnoses the updated model to drive the next round of targeted improvement. DPE has two key components. First, multiple agents annotate and quality control massive unlabeled multimodal data, using tools such as web search and image editing to produce diverse, realistic samples. Second, DPE attributes failures to specific weaknesses, dynamically adjusts the data mixture, and guides agents to generate weakness focused data for targeted reinforcement. Experiments on Qwen3-VL-8B-Instruct and Qwen2.5-VL-7B-Instruct show stable, continual gains across eleven benchmarks, indicating DPE as a scalable paradigm for continual LMM training under open task distributions. Our code, models, and data are publicly available at https://github.com/hongruijia/DPE.