Paper 2606.19534
PerceptionDLM: Parallel Region Perception with Multimodal Diffusion Language Models
- Published
- Jun 2026
- Research lab
- Independent
- Citations
- 0
- GitHub
- 77 stars
01 In brief
Summary
The paper introduces PerceptionDLM, a multimodal diffusion language model for efficient parallel region perception.
It builds on PerceptionDLM-Base, a strong diffusion-based vision-language model, and adds region prompting, RoI-aligned feature replay, and structured attention masking to generate captions for multiple image regions simultaneously in a single denoising process.
This contrasts with autoregressive models that process regions sequentially.
The authors also introduce ParaDLC-Bench, a benchmark for multi-region captioning that evaluates both quality and efficiency.
Experiments show PerceptionDLM-Base outperforms LLaDA-V on 15 of 16 multimodal benchmarks.
On ParaDLC-Bench, PerceptionDLM achieves 62.4% average accuracy, nearly doubling diffusion baselines, and offers up to 3.5x throughput speedup over AR models in dense scenarios.
The model is trained on a new ParaCaption-5.7M dataset.
The work demonstrates the potential of diffusion language models for parallel visual perception.
02 From the paper
Abstract
Multimodal large language models (MLLMs) have achieved remarkable progress in visual understanding tasks. However, most existing MLLMs rely on autoregressive generation, which limits their efficiency for perception tasks that require captioning multiple regions. In this work, we propose PerceptionDLM, a multimodal diffusion language model optimized for efficient parallel region perception. Built upon PerceptionDLM-Base, a strong foundational baseline that achieves state-of-the-art performance among open-source diffusion MLLMs, our architecture fully leverages the parallel decoding nature of DLMs. Specifically, we introduce efficient prompting and structured attention masking to enable simultaneous perception of multiple masked regions, allowing the model to generate region descriptions in parallel at both the sequence and token levels. This design significantly improves inference efficiency compared with existing approaches that process regions sequentially. To systematically evaluate the parallelism property of visual perception capability for DLMs, we construct a new Parallel Detailed Localized Captioning Benchmark (ParaDLC-Bench) by scaling the DLC-Bench to include multiple region masks per image, enabling joint evaluation of both caption quality and inference efficiency. Experiments demonstrate that PerceptionDLM maintains competitive performance in region captioning while achieving substantial speed improvements for multi-region perception tasks. Our results highlight the potential of multimodal diffusion language models for efficient, parallel visual perception. To the best of our knowledge, we are the first to achieve parallel region caption and perception by leveraging the advantages of diffusion language models. Code, models, and datasets are released.