The Year in AI Papers
The year/Independent research

Paper 2509.16197

MANZANO: A Simple and Scalable Unified Multimodal Model with a Hybrid Vision Tokenizer

Yanghao Li, Rui Qian, Bowen Pan, Haotian Zhang, Haoshuo Huang, Bowen Zhang, Jialing Tong, Haoxuan You, Xianzhi Du, Zhe Gan, Hyunjik Kim, Chao Jia, et al.

Published
Sep 2025
Research lab
Independent
Citations
15
GitHub
Not linked

01 In brief

Summary

Manzano is a unified multimodal LLM that integrates image understanding and generation using a hybrid vision tokenizer.

A single shared vision encoder feeds two lightweight adapters: a continuous adapter for understanding tasks and a discrete adapter (using FSQ with a 64K codebook) for generation, producing tokens in a common semantic space.

A unified autoregressive LLM predicts text and image tokens, and a diffusion decoder (DiT-Air) renders pixels from image tokens.

Training follows three stages: pre-training, continued pre-training, and SFT, with data mixtures including 2.3B image-text pairs, 1.7B interleaved documents, and 1B text-to-image pairs.

Ablations show the hybrid tokenizer outperforms pure-discrete and dual-encoder baselines, and unified training shows minimal task conflict.

Scaling the LLM decoder from 300M to 30B yields consistent gains across understanding and generation benchmarks.

Manzano-3B and 30B achieve state-of-the-art results among unified models, competitive with specialist models, especially on text-rich benchmarks like ChartQA, DocVQA, and OCRBench.

The model also supports image editing by conditioning both LLM and diffusion decoder on a reference image.

02 From the paper

Abstract

Unified multimodal Large Language Models (LLMs) that can both understand and generate visual content hold immense potential. However, existing open-source models often suffer from a performance trade-off between these capabilities. We present Manzano, a simple and scalable unified framework that substantially reduces this tension by coupling a hybrid image tokenizer with a well-curated training recipe. A single shared vision encoder feeds two lightweight adapters that produce continuous embeddings for image-to-text understanding and discrete tokens for text-to-image generation within a common semantic space. A unified autoregressive LLM predicts high-level semantics in the form of text and image tokens, with an auxiliary diffusion decoder subsequently translating the image tokens into pixels. The architecture, together with a unified training recipe over understanding and generation data, enables scalable joint learning of both capabilities. Manzano achieves state-of-the-art results among unified models, and is competitive with specialist models, particularly on text-rich evaluation. Our studies show minimal task conflicts and consistent gains from scaling model size, validating our design choice of a hybrid tokenizer.