Paper 2604.24763
Tuna-2: Pixel Embeddings Beat Vision Encoders for Multimodal Understanding and Generation
01 In brief
Summary
Tuna-2 is a native unified multimodal model that performs visual understanding and generation directly from raw pixel embeddings, eliminating pretrained vision encoders such as VAEs and representation encoders.
It uses simple patch embedding layers to encode images and a single transformer decoder for joint processing, with pixel-space flow matching for generation.
A masking-based feature learning scheme stabilizes training and improves robustness.
Tuna-2 achieves state-of-the-art results on multimodal understanding benchmarks, outperforming encoder-based variants like Tuna-R on fine-grained perception tasks, while remaining competitive on generation benchmarks like GenEval and DPG-Bench.
It also shows strong image editing and reconstruction performance.
Controlled comparisons reveal that while encoder-based models converge faster early, the encoder-free design scales better with data, and attention visualizations show more accurate cross-modal alignment.
The training pipeline includes two end-to-end stages: pretraining on image captioning and text-to-image generation, followed by supervised finetuning on instruction-following, editing, and high-quality generation data.
02 From the paper
Abstract
Unified multimodal models typically rely on pretrained vision encoders and use separate visual representations for understanding and generation, creating misalignment between the two tasks and preventing fully end-to-end optimization from raw pixels. We introduce Tuna-2, a native unified multimodal model that performs visual understanding and generation directly based on pixel embeddings. Tuna-2 drastically simplifies the model architecture by employing simple patch embedding layers to encode visual input, completely discarding the modular vision encoder designs such as the VAE or the representation encoder. Experiments show that Tuna-2 achieves state-of-the-art performance in multimodal benchmarks, demonstrating that unified pixel-space modelling can fully compete with latent-space approaches for high-quality image generation. Moreover, while the encoder-based variant converges faster in early pretraining, Tuna-2's encoder-free design achieves stronger multimodal understanding at scale, particularly on tasks requiring fine-grained visual perception. These results show that pretrained vision encoders are not necessary for multimodal modelling, and end-to-end pixel-space learning offers a scalable path toward stronger visual representations for both generation and perception.