The Year in AI Papers
The year/Independent research

Paper 2602.08676

LLaDA2.1: Speeding Up Text Diffusion via Token Editing

Tiwei Bie, Maosong Cao, Xiang Cao, Bingsen Chen, Fuyuan Chen, Kun Chen, Lun Du, Daozhuo Feng, Haibo Feng, Mingliang Gong, Zhuocheng Gong, Yanmei Gu, et al.

Published
Feb 2026
Research lab
Independent
Citations
38
GitHub
501 stars

01 In brief

Summary

LLaDA2.1 introduces a novel decoding scheme for discrete diffusion language models (dLLMs) that combines Mask-to-Token (M2T) and Token-to-Token (T2T) editing, controlled by dual probability thresholds.

This allows two operating modes: Speedy Mode (S Mode) lowers the M2T threshold for faster generation, relying on T2T to correct errors, while Quality Mode (Q Mode) uses conservative thresholds for higher benchmark performance.

The model is trained with a mixture of M2T and T2T objectives during CPT and SFT, and uses a new ELBO-based Block-level Policy Optimization (EBPO) for reinforcement learning, enabling stable gradient estimation.

LLaDA2.1 releases two models: LLaDA2.1-Mini (16B) and LLaDA2.1-Flash (100B).

Across 33 benchmarks, LLaDA2.1 achieves strong performance and high decoding speed, with the Flash model reaching 892 TPS on HumanEval+, 801 TPS on BigCodeBench, and 663 TPS on LiveCodeBench.

Multi-Block Editing (MBE) further improves performance on reasoning and coding tasks with modest throughput reduction.

The paper also discusses infrastructure optimizations like Alpha-MoE and FP8 quantization, and notes a tradeoff between speed and accuracy, recommending S Mode for structured tasks and Q Mode for general chat.

02 From the paper

Abstract

While LLaDA2.0 showcased the scaling potential of 100B-level block-diffusion models and their inherent parallelization, the delicate equilibrium between decoding speed and generation quality has remained an elusive frontier. Today, we unveil LLaDA2.1, a paradigm shift designed to transcend this trade-off. By seamlessly weaving Token-to-Token (T2T) editing into the conventional Mask-to-Token (M2T) scheme, we introduce a joint, configurable threshold-decoding scheme. This structural innovation gives rise to two distinct personas: the Speedy Mode (S Mode), which audaciously lowers the M2T threshold to bypass traditional constraints while relying on T2T to refine the output; and the Quality Mode (Q Mode), which leans into conservative thresholds to secure superior benchmark performances with manageable efficiency degrade. Furthering this evolution, underpinned by an expansive context window, we implement the first large-scale Reinforcement Learning (RL) framework specifically tailored for dLLMs, anchored by specialized techniques for stable gradient estimation. This alignment not only sharpens reasoning precision but also elevates instruction-following fidelity, bridging the chasm between diffusion dynamics and complex human intent. We culminate this work by releasing LLaDA2.1-Mini (16B) and LLaDA2.1-Flash (100B). Across 33 rigorous benchmarks, LLaDA2.1 delivers strong task performance and lightning-fast decoding speed. Despite its 100B volume, on coding tasks it attains an astounding 892 TPS on HumanEval+, 801 TPS on BigCodeBench, and 663 TPS on LiveCodeBench.