The Year in AI Papers
The year/Independent research

Paper 2605.31604

Representation Forcing for Bottleneck-Free Unified Multimodal Models

Yuqing Wang, Zhijie Lin, Ceyuan Yang, Yang Zhao, Fei Xiao, Hao He, Qi Zhao, Zihan Ding, Fuyun Wang, Shuai Wang, Youliang Zhang, Haoqi Fan, et al.

Published
May 2026
Research lab
Independent
Citations
3
GitHub
Not linked

01 In brief

Summary

The paper introduces Representation Forcing (RF), a technique for unified multimodal models (UMMs) that eliminates the need for a separately pretrained VAE in image generation.

RF trains the decoder to autoregressively predict discrete visual representation tokens, derived from the model's own understanding encoder via online vector quantization, before generating pixels.

These tokens remain in context and guide pixel-space diffusion within the same transformer backbone.

Experiments show that a pixel-space model with RF matches VAE-based counterparts on text-to-image benchmarks (GenEval 0.84, DPG-Bench 84.15) and outperforms them on understanding tasks (6/8 benchmarks).

Ablations show RF is critical for pixel-space generation (GenEval 0.76 vs 0.25 without), outperforms auxiliary alignment (REPA) and continuous regression, and benefits both pixel-space and VAE-based settings.

The work advocates for end-to-end, bottleneck-free UMMs where perception and generation share a single learned representation space.

02 From the paper

Abstract

Unified multimodal models (UMMs) aim to handle perception and generation in a single model. Yet existing UMMs still rely on a frozen, separately pretrained VAE for image generation, imposing a structural bottleneck. Naively removing it introduces a quality gap, as the model must learn both high-level structure and low-level details from raw pixels. In this paper, we propose Representation Forcing (RF), a technique that closes this gap by making representation prediction a native capability of the model. Concretely, RF forces the decoder to autoregressively predict visual representations as intermediate tokens before pixels; these tokens then stay in context to guide pixel diffusion within the same backbone. By turning representations from perception outputs into generation targets, RF eliminates the need for any external generative latent space. We find that RF benefits both understanding and generation. On image generation, our pixel-space model with RF matches state-of-the-art VAE-based unified models. On image understanding, pixel-space RF generally outperforms its VAE-based variant. Together, these results offer an effective step toward end-to-end, bottleneck-free UMMs.