Research collection

Diffusion LMs and decoding

Diffusion and non-autoregressive language models, parallel and speculative decoding, and sampling or decoding strategies.

Papers: 17
Research labs: 2
Official code: 11

1–17 of 17 papers in this collection

01

Independent research

PerceptionDLM: Parallel Region Perception with Multimodal Diffusion Language Models

The paper introduces PerceptionDLM, a multimodal diffusion language model for efficient parallel region perception. It builds on PerceptionDLM-Base, a strong diffusion-based vision-language model, and adds region prompting, RoI-aligned feature replay, and structured attention masking to generate captions for multiple image regions simultaneously in a…

Yueyi Sun, Yuhao Wang, Jason Li, Ye Tian, et al.

Published: Jun 2026
Citations: 0
Code: 77 stars

Read summary Original

02

arXiv.org

Continuous Latent Diffusion Language Model

Cola DLM is a hierarchical latent diffusion language model that decomposes text generation into global semantic modeling in a continuous latent space and local textual realization via a conditional decoder. It uses a Text VAE to map text to latent variables, a block-causal DiT to model the latent prior via flow matching, and a decoder for text generation.…

Hongcan Guo, Qinyu Zhao, Yian Zhao, Shen Nie, et al.

Published: May 2026
Citations: 4
Code: 271 stars

Read summary Original

03

arXiv.org

LLaDA2.0-Uni: Unifying Multimodal Understanding and Generation with Diffusion Large Language Model

LLaDA2.0-Uni is a unified discrete diffusion large language model (dLLM) that integrates multimodal understanding and generation in a single framework. Its architecture combines a SigLIP-VQ semantic tokenizer, a 16B MoE dLLM backbone, and a diffusion decoder. The tokenizer converts images into discrete semantic tokens, enabling block-level masked diffusion…

Inclusion AI, Tiwei Bie, Haoxing Chen, Tieyuan Chen, et al.

Published: Apr 2026
Citations: 5
Code: 772 stars

Read summary Original

04

arXiv.org

DMax: Aggressive Parallel Decoding for dLLMs

DMax is a new paradigm for diffusion language models (dLLMs) that enables aggressive parallel decoding while preserving generation quality by mitigating error accumulation. It reformulates decoding as a progressive self-refinement from mask embeddings to token embeddings, using two key components: On-Policy Uniform Training (OPUT) and Soft Parallel…

Zigeng Chen, Gongfan Fang, Xinyin Ma, Ruonan Yu, et al.

Published: Apr 2026
Citations: 8
Code: 128 stars

Read summary Original

05

Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)

dLLM: Simple Diffusion Language Modeling

The paper introduces dLLM, an open-source framework that standardizes the training, inference, and evaluation of diffusion language models (DLMs). It provides modular trainers for Masked Diffusion (MDLM) and Block Diffusion (BD3LM), a plug-and-play sampler abstraction supporting efficient decoding like Fast-dLLM, and a unified evaluation pipeline that…

Zhanhui Zhou, Lingjie Chen, Hanghang Tong, Dawn Song

Published: Feb 2026
Citations: 12
Code: Not linked

Read summary Original

06

arXiv.org

MolHIT: Advancing Molecular-Graph Generation with Hierarchical Discrete Diffusion Models

MolHIT is a molecular graph generation framework based on a Hierarchical Discrete Diffusion Model (HDDM) that generalizes discrete diffusion by adding mid-level states encoding chemical priors, and a Decoupled Atom Encoding (DAE) that splits atom types by chemical roles (aromaticity, formal charge). This addresses limitations in prior graph diffusion…

Hojung Jung, Rodrigo Hormazabal, Jaehyeong Jo, Youngrok Park, et al.

Published: Feb 2026
Citations: 0
Code: Not linked

Read summary Original

07

arXiv.org

LLaDA2.1: Speeding Up Text Diffusion via Token Editing

LLaDA2.1 introduces a novel decoding scheme for discrete diffusion language models (dLLMs) that combines Mask-to-Token (M2T) and Token-to-Token (T2T) editing, controlled by dual probability thresholds. This allows two operating modes: Speedy Mode (S Mode) lowers the M2T threshold for faster generation, relying on T2T to correct errors, while Quality Mode…

Tiwei Bie, Maosong Cao, Xiang Cao, Bingsen Chen, et al.

Published: Feb 2026
Citations: 38
Code: 501 stars

Read summary Original

08

Independent research

The Flexibility Trap: Rethinking the Value of Arbitrary Order in Diffusion Language Models

This paper challenges the assumption that arbitrary-order generation in Diffusion Large Language Models (dLLMs) enhances reasoning potential. The authors find that for general reasoning tasks like mathematics and coding, arbitrary-order generation can actually limit reasoning potential compared to standard autoregressive (AR) order. They introduce the…

Zanlin Ni, Shenzhi Wang, Yang Yue, Tianyu Yu, et al.

Published: Jan 2026
Citations: 2
Code: 259 stars

Read summary Original

09

arXiv.org

ReFusion: A Diffusion Large Language Model with Parallel Autoregressive Decoding

ReFusion is a masked diffusion language model that integrates sequence reorganization into a causal attention framework to overcome the limitations of standard masked diffusion models (MDMs), which suffer from high computational overhead due to the lack of Key-Value (KV) caching and incoherent generation from intractable token combination learning.…

Jia-Nan Li, Jian Guan, Wei Wu, Chongxuan Li

Published: Dec 2025
Citations: 14
Code: 63 stars

Read summary Original

10

arXiv.org

LLaDA2.0: Scaling Up Diffusion Language Models to 100B

LLaDA2.0 introduces a family of discrete diffusion language models (dLLMs) scaled up to 100B parameters by converting pre-trained auto-regressive (AR) models, avoiding costly training from scratch. The conversion uses a three-phase Warmup-Stable-Decay (WSD) continual pre-training strategy: progressively increasing block size in block diffusion (warmup),…

Tiwei Bie, Maosong Cao, Kun Chen, Lun Du, et al.

Published: Dec 2025
Citations: 123
Code: 501 stars

Read summary Original

11

Together AI

CDLM: Consistency Diffusion Language Models for Faster Sampling

CDLM (Consistency Diffusion Language Models) is a training-based method to accelerate inference in Diffusion Language Models (DLMs) by addressing two bottlenecks: excessive refinement steps and incompatibility with KV caching. It integrates consistency modeling to reduce sampling steps via multi-token finalization and enforces a block-wise causal attention…

Minseo Kim, Chenfeng Xu, Coleman Hooper, Harman M. Singh, et al.

Published: Nov 2025
Citations: 17
Code: 41 stars

Read summary Original

12

arXiv.org

MMaDA-Parallel: Multimodal Large Diffusion Language Models for Thinking-Aware Editing and Generation

The paper introduces MMaDA-Parallel, a parallel multimodal diffusion framework for thinking-aware image editing and generation. The authors identify a failure mode in sequential autoregressive approaches where pre-generation reasoning can degrade performance due to error propagation. They propose ParaBench, a benchmark evaluating both text and image…

Ye Tian, Ling Yang, Jiongfan Yang, Anran Wang, et al.

Published: Nov 2025
Citations: 8
Code: 306 stars

Read summary Original

13

NVIDIA

TiDAR: Think in Diffusion, Talk in Autoregression

TiDAR is a sequence-level hybrid architecture that combines diffusion and autoregressive (AR) language modeling in a single forward pass using structured attention masks. It drafts tokens in parallel via diffusion (thinking) and samples final outputs autoregressively (talking), exploiting free GPU compute slots to achieve high throughput without quality…

Jingyu Liu, Xin Dong, Zhifan Ye, Rishabh Mehta, et al.

Published: Nov 2025
Citations: 33
Code: Not linked

Read summary Original

14

arXiv.org

The Principles of Diffusion Models

This book provides a comprehensive, mathematically grounded introduction to diffusion models, tracing their origins from three foundational perspectives: variational (VAEs to DDPMs), score-based (EBMs to NCSN and Score SDEs), and flow-based (Normalizing Flows to Flow Matching). It unifies these views through the Fokker-Planck equation and a common…

Chieh-Hsin Lai, Yang Song, Dongjun Kim, Yuki Mitsufuji, et al.

Published: Oct 2025
Citations: 69
Code: Not linked

Read summary Original

15

arXiv.org

Latent Refinement Decoding: Enhancing Diffusion-Based Language Models by Refining Belief States

The paper introduces Latent Refinement Decoding (LRD), a two-stage decoding framework for diffusion-based language models (dLLMs) that addresses information loss and premature commitment in existing hard-assignment methods. LRD operates in two phases: Phase 1 (Latent Refinement) performs distribution-preserving updates in embedding space by mixing [MASK]…

Qinglin Zhu, Yizhen Yao, Runcong Zhao, Yanzheng Xiang, et al.

Published: Oct 2025
Citations: 9
Code: Not linked

Read summary Original

16

NVIDIA

Fast-dLLM v2: Efficient Block-Diffusion LLM

Fast-dLLM v2 is a block diffusion language model that adapts pretrained autoregressive LLMs (Qwen2.5-Instruct 1.5B and 7B) for parallel text generation. It requires only ~1B tokens of fine-tuning, a 500x reduction compared to full-attention diffusion models like Dream (580B tokens). The method uses block-wise diffusion with complementary masking and a…

Chengyue Wu, Hao Zhang, Shuchen Xue, Shizhe Diao, et al.

Published: Sep 2025
Citations: 114
Code: 1.1K stars

Read summary Original

17

arXiv.org

Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference

Seed Diffusion Preview is a large-scale discrete-state diffusion language model developed by ByteDance Seed and Tsinghua AIR, focused on code generation. It achieves an inference speed of 2,146 tokens per second on H20 GPUs, significantly faster than contemporary models like Mercury Coder and Gemini Diffusion, while maintaining competitive performance on…

Yuxuan Song, Zheng Zhang, Cheng Luo, Pengyang Gao, et al.

Published: Aug 2025
Citations: 163
Code: Not linked

Read summary Original