Paper 2602.22661
dLLM: Simple Diffusion Language Modeling
- Published
- Feb 2026
- Research lab
- Independent
- Citations
- 12
- GitHub
- Not linked
01 In brief
Summary
The paper introduces dLLM, an open-source framework that standardizes the training, inference, and evaluation of diffusion language models (DLMs).
It provides modular trainers for Masked Diffusion (MDLM) and Block Diffusion (BD3LM), a plug-and-play sampler abstraction supporting efficient decoding like Fast-dLLM, and a unified evaluation pipeline that reproduces official results.
The framework enables finetuning large DLMs (e.g., LLaDA, Dream) and converting existing models into DLMs with minimal compute.
Recipes and checkpoints are released for converting BERT-style encoders (ModernBERT) and autoregressive LMs (Qwen3-0.6B) into DLMs via SFT, showing competitive performance on several benchmarks.
Key findings include that DLM evaluation is highly sensitive to inference hyperparameters, and that lightweight conversion of pretrained models is a practical path to building DLMs.
The framework is available at GitHub and Hugging Face hubs, aiming to lower the barrier for DLM research and development.
Future work includes integrating RL algorithms and supporting more models.
The paper also includes detailed evaluation reproduction tables and training curves in appendices, demonstrating the framework's consistency with official implementations and its speedups in inference.
Overall, dLLM addresses the lack of unified tools in the DLM field, promoting reproducibility and accessibility.
The authors acknowledge support from the Berkeley Fellowship.
The work is under review as a preprint.
The framework's modular design allows easy extension to new methods, and the released checkpoints provide baselines for future research.
The paper emphasizes the importance of standardizing components to accelerate progress in diffusion language modeling.
The evaluation results show that dLLM closely matches official scores, validating its…
02 From the paper
Abstract
Although diffusion language models (DLMs) are evolving quickly, many recent models converge on a set of shared components. These components, however, are distributed across ad-hoc research codebases or lack transparent implementations, making them difficult to reproduce or extend. As the field accelerates, there is a clear need for a unified framework that standardizes these common components while remaining flexible enough to support new methods and architectures. To address this gap, we introduce dLLM, an open-source framework that unifies the core components of diffusion language modeling -- training, inference, and evaluation -- and makes them easy to customize for new designs. With dLLM, users can reproduce, finetune, deploy, and evaluate open-source large DLMs such as LLaDA and Dream through a standardized pipeline. The framework also provides minimal, reproducible recipes for building small DLMs from scratch with accessible compute, including converting any BERT-style encoder or autoregressive LM into a DLM. We also release the checkpoints of these small DLMs to make DLMs more accessible and accelerate future research.