Paper 2509.25541
Vision-Zero: Scalable VLM Self-Improvement via Strategic Gamified Self-Play
- Published
- Sep 2025
- Research lab
- Independent
- Citations
- 33
- GitHub
- 165 stars
01 In brief
Summary
Vision-Zero is a label-free, domain-agnostic multi-agent self-play framework for self-evolving vision-language models (VLMs) through competitive visual games generated from arbitrary images.
It trains VLMs in a 'Who Is the Spy?'-style game where civilians see an image and the spy sees a blank input, requiring strategic reasoning and communication.
The framework uses Iterative Self-Play Policy Optimization (Iterative-SPO), which alternates between self-play and reinforcement learning with verifiable rewards (RLVR) to avoid performance plateaus.
Vision-Zero accepts arbitrary images (CLEVR scenes, charts, real-world images) and requires no human annotation.
Experiments on Qwen2.5-VL-7B, InternVL3-8B, and InternVL3-14B show state-of-the-art performance on reasoning, chart QA, and vision-centric tasks, surpassing baselines trained on human-labeled data.
It also reduces training costs (e.g., 127 A100-hours vs.
700 for MM-Eureka) and improves training efficiency by 3.3x to 6.4x over standard GRPO.
The framework mitigates negative transfer and enhances reasoning, spatial understanding, and visual comprehension.
Models and code are released publicly.
Key contributions include the first gamified self-play framework for VLMs with zero-human-in-the-loop post-training, the Iterative-SPO algorithm, and demonstrated performance gains across diverse tasks.
Limitations include reliance on image editors for generating image pairs and current design for single-image observations, with future work extending to videos and 3D environments.
The paper was published at ICLR 2026 and is available on GitHub.
The work was conducted by researchers from Duke University, National University of Singapore, University of Maryland, and Adobe Inc.
The framework's strategic environment and domain-agnostic inputs enable scalable self-improvement without human supervision, addressing data scarcity and knowledge ceiling issues in VLM…
02 From the paper
Abstract
Although reinforcement learning (RL) has emerged as a promising approach for improving vision-language models (VLMs) and multimodal large language models (MLLMs), current methods rely heavily on manually curated datasets and costly human verification, which limits scalable self-improvement in multimodal systems. To address this challenge, we propose Vision-Zero, a label-free, domain-agnostic multi-agent self-play framework for self-evolving VLMs through competitive visual games generated from arbitrary image inputs. Specifically, Vision-Zero encompasses three main attributes: (1) Strategic Self-Play Framework: Vision-Zero trains VLMs in "Who Is the Spy"-style games, where the models engage in strategic reasoning and actions across multiple roles. Through interactive gameplay, models autonomously generate their training data without human annotation. (2) Gameplay from Arbitrary Images: Unlike existing gamified frameworks, Vision-Zero can generate games from arbitrary images, thereby enhancing the model's reasoning ability across diverse domains and showing strong generalization to different tasks. We demonstrate this versatility using three distinct types of image datasets: CLEVR-based synthetic scenes, charts, and real-world images. (3) Sustainable Performance Gain: We introduce Iterative Self-Play Policy Optimization (Iterative-SPO), a novel training algorithm that alternates between Self-Play and reinforcement learning with verifiable rewards (RLVR), mitigating the performance plateau often seen in self-play-only training and achieving sustained long-term improvements. Despite using label-free data, Vision-Zero achieves state-of-the-art performance on reasoning, chart question answering, and vision-centric understanding tasks, surpassing other annotation-based methods. Models and code have been released at https://github.com/wangqinsi1/Vision-Zero.