The Year in AI Papers
The year/Meta AI

Paper 2604.04746

Think in Strokes, Not Pixels: Process-Driven Image Generation via Interleaved Reasoning

Lei Zhang, Junjiao Tian, Zhipeng Fan, Kunpeng Li, Jialiang Wang, Weifeng Chen, Markos Georgopoulos, Felix Juefei-Xu, Yuxiang Bao, Julian McAuley, Manling Li, Zecheng He

Published
Apr 2026
Research lab
Meta AI
Citations
3
GitHub
Not linked

01 In brief

Summary

This paper introduces process-driven image generation, a multi-step paradigm that decomposes text-to-image synthesis into an interleaved reasoning trajectory of textual planning and visual generation.

The method uses a recurring four-stage cycle: Plan, Sketch, Inspect, and Refine, where the model generates incremental instructions and scene descriptions, synthesizes partial drafts, detects conflicts, and issues refinements.

To supervise ambiguous intermediate states, the authors construct a dataset with three components: multi-turn generation via scene-graph subsampling, instruction-intermediate conflict reasoning via self-sampling, and image-instruction alignment from Gen-Ref.

They fine-tune BAGEL-7B end-to-end to autoregressively emit interleaved text and image tokens.

Experiments show improvements on GenEval (0.79 to 0.83) and WISE (0.70 to 0.76), outperforming PARM with 8x less inference cost and 11x less training data.

Ablations reveal that diverse editing instructions and self-sampled critiques are key, outperforming symbolic corrections by +6%.

02 From the paper

Abstract

Humans paint images incrementally: they plan a global layout, sketch a coarse draft, inspect, and refine details, and most importantly, each step is grounded in the evolving visual states. However, can unified multimodal models trained on text-image interleaved datasets also imagine the chain of intermediate states? In this paper, we introduce process-driven image generation, a multi-step paradigm that decomposes synthesis into an interleaved reasoning trajectory of thoughts and actions. Rather than generating images in a single step, our approach unfolds across multiple iterations, each consisting of 4 stages: textual planning, visual drafting, textual reflection, and visual refinement. The textual reasoning explicitly conditions how the visual state should evolve, while the generated visual intermediate in turn constrains and grounds the next round of textual reasoning. A core challenge of process-driven generation stems from the ambiguity of intermediate states: how can models evaluate each partially-complete image? We address this through dense, step-wise supervision that maintains two complementary constraints: for the visual intermediate states, we enforce the spatial and semantic consistency; for the textual intermediate states, we preserve the prior visual knowledge while enabling the model to identify and correct prompt-violating elements. This makes the generation process explicit, interpretable, and directly supervisable. To validate proposed method, we conduct experiments under various text-to-image generation benchmarks.