Paper 2509.26328
Fast-dLLM v2: Efficient Block-Diffusion LLM
- Published
- Sep 2025
- Research lab
- NVIDIA
- Citations
- 114
- GitHub
- 1.1K stars
01 In brief
Summary
Fast-dLLM v2 is a block diffusion language model that adapts pretrained autoregressive LLMs (Qwen2.5-Instruct 1.5B and 7B) for parallel text generation.
It requires only ~1B tokens of fine-tuning, a 500x reduction compared to full-attention diffusion models like Dream (580B tokens).
The method uses block-wise diffusion with complementary masking and a shifted-label strategy to preserve AR training objectives.
A hierarchical caching mechanism (block-level and sub-block DualCache) enables efficient decoding.
Fast-dLLM v2 achieves up to 2.5x speedup over standard AR decoding without quality loss.
On benchmarks, the 7B model scores 60.3 average, surpassing Qwen2.5-7B-Nemo-FT (59.6) and Dream (57.6).
With a confidence threshold of 0.9, throughput increases from 39.1 to 101.7 tokens/s (2.6x speedup) on GSM8K.
Ablations show that padding and complementary masking improve average accuracy by +3.7 points, and sub-block size 8 is optimal.
Training used 64 A100 GPUs: 1.5B model for 6,000 steps (8 hours), 7B for 2,500 steps (12 hours).
02 From the paper
Abstract
Autoregressive (AR) large language models (LLMs) have achieved remarkable performance across a wide range of natural language tasks, yet their inherent sequential decoding limits inference efficiency. In this work, we propose Fast-dLLM v2, a carefully designed block diffusion language model (dLLM) that efficiently adapts pretrained AR models into dLLMs for parallel text generation, requiring only approximately 1B tokens of fine-tuning. This represents a 500x reduction in training data compared to full-attention diffusion LLMs such as Dream (580B tokens), while preserving the original model's performance. Our approach introduces a novel training recipe that combines a block diffusion mechanism with a complementary attention mask, enabling blockwise bidirectional context modeling without sacrificing AR training objectives. To further accelerate decoding, we design a hierarchical caching mechanism: a block-level cache that stores historical context representations across blocks, and a sub-block cache that enables efficient parallel generation within partially decoded blocks. Coupled with our parallel decoding pipeline, Fast-dLLM v2 achieves up to 2.5x speedup over standard AR decoding without compromising generation quality. Extensive experiments across diverse benchmarks demonstrate that Fast-dLLM v2 matches or surpasses AR baselines in accuracy, while delivering state-of-the-art efficiency among dLLMs - marking a significant step toward the practical deployment of fast and accurate LLMs. Code and model will be publicly released.