The Year in AI Papers
The year/Independent research

Paper 2508.10711

NextStep-1: Toward Autoregressive Image Generation with Continuous Tokens at Scale

NextStep Team, Chunrui Han, Guopeng Li, Jingwei Wu, Quan Sun, Yan Cai, Yuang Peng, Zheng Ge, Deyu Zhou, Haomiao Tang, Hongyu Zhou, Kenkun Liu, et al.

Published
Aug 2025
Research lab
Independent
Citations
52
GitHub
694 stars

01 In brief

Summary

NextStep-1 is a 14B autoregressive model for text-to-image generation that uses continuous image tokens instead of discrete VQ tokens, paired with a 157M flow matching head.

It is initialized from Qwen2.5-14B and trained with next-token prediction, combining cross-entropy loss for text and flow matching loss for images.

The image tokenizer, fine-tuned from Flux VAE, uses 16-channel latents with channel-wise normalization and stochastic perturbation to stabilize training.

The model achieves state-of-the-art results among autoregressive models, scoring 0.63 on GenEval, 0.88 on GenAI-Bench basic, 0.67 on advanced, 85.28 on DPG-Bench, 0.54 on WISE, and 0.417 on OneIG-Bench.

For image editing, NextStep-1-Edit scores 6.58 on GEdit-Bench-EN and 3.71 on ImgEdit-Bench.

Training includes pre-training (stages 1, 2, annealing) and post-training (SFT, DPO).

Key findings: the transformer backbone is the core generative model, the tokenizer's noise regularization improves generation quality, and high-dimensional latents can cause artifacts.

Limitations include inference latency and challenges in high-resolution training and SFT stability.

02 From the paper

Abstract

Prevailing autoregressive (AR) models for text-to-image generation either rely on heavy, computationally-intensive diffusion models to process continuous image tokens, or employ vector quantization (VQ) to obtain discrete tokens with quantization loss. In this paper, we push the autoregressive paradigm forward with NextStep-1, a 14B autoregressive model paired with a 157M flow matching head, training on discrete text tokens and continuous image tokens with next-token prediction objectives. NextStep-1 achieves state-of-the-art performance for autoregressive models in text-to-image generation tasks, exhibiting strong capabilities in high-fidelity image synthesis. Furthermore, our method shows strong performance in image editing, highlighting the power and versatility of our unified approach. To facilitate open research, we will release our code and models to the community.