The Year in AI Papers
The year/Independent research

Paper 2604.27505

Leveraging Verifier-Based Reinforcement Learning in Image Editing

Hanzhong Guo, Jie Wu, Jie Liu, Yu Gao, Zilyu Ye, Linxiao Yuan, Xionghui Wang, Yizhou Yu, Weilin Huang

Published
Apr 2026
Research lab
Independent
Citations
5
GitHub
Not linked

01 In brief

Summary

The paper introduces Edit-R1, a framework for image editing that uses a verifier-based Reasoning Reward Model (RRM) trained with reinforcement learning.

The RRM decomposes editing instructions into principles, evaluates edited images against each, and produces interpretable scores.

Training involves a cold-start SFT phase with filtered chain-of-thought data and a novel Group Contrastive Preference Optimization (GCPO) algorithm that aligns the pointwise RRM with pairwise human preferences.

The RRM is then used as a reward signal in GRPO to optimize editing models.

The 7B RRM achieves 82.2% accuracy on an internal benchmark, surpassing Seed-1.5-VL (79.3%) and EditScore-7B (65.9%).

Applying Edit-R1 to FLUX.Kontext improves its overall score from 5.77 to 6.24 and semantic consistency from 6.27 to 6.86, with a +23.2 GSB human evaluation score.

For Qwen-Edit, it yields a 15.2% relative gain in the Motion Change category.

The framework demonstrates scalability from 3B to 7B parameters and effective human alignment with only 10k preference pairs.

02 From the paper

Abstract

While Reinforcement Learning from Human Feedback (RLHF) has become a pivotal paradigm for text-to-image generation, its application to image editing remains largely unexplored. A key bottleneck is the lack of a robust general reward model for all editing tasks. Existing edit reward models usually give overall scores without detailed checks, ignoring different instruction requirements and causing biased rewards. To address this, we argue that the key is to move from a simple scorer to a reasoning verifier. We introduce Edit-R1, a framework that builds a chain-of-thought (CoT) verifier-based reasoning reward model (RRM) and then leverages it for downstream image editing. The Edit-RRM breaks instructions into distinct principles, evaluates the edited image against each principle, and aggregates these checks into an interpretable, fine-grained reward. To build such an RRM, we first apply supervised fine-tuning (SFT) as a ``cold-start'' to generate CoT reward trajectories. Then, we introduce Group Contrastive Preference Optimization (GCPO), a reinforcement learning algorithm that leverages human pairwise preference data to reinforce our pointwise RRM. After building the RRM, we use GRPO to train editing models with this non-differentiable yet powerful reward model. Extensive experiments demonstrate that our Edit-RRM surpasses powerful VLMs such as Seed-1.5-VL and Seed-1.6-VL as an editing-specific reward model, and we observe a clear scaling trend, with performance consistently improving from 3B to 7B parameters. Moreover, Edit-R1 delivers gains to editing models like FLUX.1-kontext, highlighting its effectiveness in enhancing image editing.