The Year in AI Papers
The year/Independent research

Paper 2603.03143

Edit in 2D, Verify in 3D: Reinforcement Learning for Multi-view Consistent Scene Editing

Jiyuan Wang, Chunyu Lin, Lei Sun, Zhi Cao, Yuyang Yin, Lang Nie, Zhenlong Yuan, Xiangxiang Chu, Yunchao Wei, Kang Liao, Guosheng Lin

Published
Mar 2026
Research lab
Independent
Citations
7
GitHub
203 stars

01 In brief

Summary

The paper introduces RL3DEdit, a single-pass framework for multi-view consistent 3D scene editing that uses reinforcement learning (RL) to augment a 2D diffusion editor (FLUX-Kontext) with 3D consistency priors.

The authors observe that while generating multi-view consistent images is challenging, verifying consistency is tractable, making RL a suitable solution.

They leverage the 3D foundation model VGGT as a verifier, using its confidence maps and pose predictions as reward signals.

The method employs GRPO optimization with rewards for depth/point confidence, relative pose alignment, and an anchor-based quality reward.

RL3DEdit requires no per-scene fine-tuning, handles geometry-changing edits, and achieves state-of-the-art results with over 2x speedup compared to existing methods.

Experiments show superior editing quality and multi-view consistency, with zero-shot generalization to unseen instructions and scenes.

The framework is also transferable to other 2D editors like Qwen-Image-Edit.

Limitations include dependence on the backbone's context length and challenges with extreme non-rigid deformations.

The code and model will be released.

02 From the paper

Abstract

Leveraging the priors of 2D diffusion models for 3D editing has emerged as a promising paradigm. However, multi-view consistency remains challenging in edited results, and the extreme scarcity of paired 3D-consistent editing data makes supervised fine-tuning (SFT) impractical, despite its effectiveness for editing tasks. In this paper, we observe that, while generating multi-view consistent 3D content is highly challenging, verifying 3D consistency is tractable, naturally positioning reinforcement learning (RL) as a feasible solution. Motivated by this, we propose RL3DEdit, a single-pass framework driven by RL optimization with novel rewards derived from the 3D foundation model, VGGT. Specifically, we leverage VGGT's robust priors learned from massive real-world data, feed the edited images into it, and utilize the output confidence maps and pose estimation errors as reward signals, effectively anchoring the 2D editing priors onto a 3D-consistent manifold via RL. Extensive experiments demonstrate that RL3DEdit achieves stable multi-view consistency and outperforms state-of-the-art methods in editing quality with high efficiency. To promote the development of 3D editing, we will release the code and model.