Paper 2511.19365
DeCo: Frequency-Decoupled Pixel Diffusion for End-to-End Image Generation
- Published
- Nov 2025
- Research lab
- Independent
- Citations
- 37
- GitHub
- 238 stars
01 In brief
Summary
DeCo is a frequency-decoupled pixel diffusion framework for end-to-end image generation.
It addresses the challenge of pixel diffusion models jointly modeling high-frequency signals and low-frequency semantics in a single diffusion transformer (DiT), which slows training and inference.
DeCo uses a DiT to model low-frequency semantics from downsampled inputs, while a lightweight, attention-free pixel decoder generates high-frequency details conditioned on the DiT's semantic guidance.
A frequency-aware flow-matching loss, inspired by JPEG quantization tables, reweights frequency components to emphasize visually salient ones.
Experiments show DeCo achieves FID of 1.62 (256×256) and 2.22 (512×512) on ImageNet, closing the gap with latent diffusion methods, and a leading GenEval score of 0.86 for text-to-image generation.
The framework also improves training efficiency, reaching a FID of 2.57 in 400k iterations, 10× faster than the baseline.
02 From the paper
Abstract
Pixel diffusion aims to generate images directly in pixel space in an end-to-end fashion. This approach avoids the limitations of VAE in the two-stage latent diffusion, offering higher model capacity. Existing pixel diffusion models suffer from slow training and inference, as they usually model both high-frequency signals and low-frequency semantics within a single diffusion transformer (DiT). To pursue a more efficient pixel diffusion paradigm, we propose the frequency-DeCoupled pixel diffusion framework. With the intuition to decouple the generation of high and low frequency components, we leverage a lightweight pixel decoder to generate high-frequency details conditioned on semantic guidance from the DiT. This thus frees the DiT to specialize in modeling low-frequency semantics. In addition, we introduce a frequency-aware flow-matching loss that emphasizes visually salient frequencies while suppressing insignificant ones. Extensive experiments show that DeCo achieves superior performance among pixel diffusion models, attaining FID of 1.62 (256x256) and 2.22 (512x512) on ImageNet, closing the gap with latent diffusion methods. Furthermore, our pretrained text-to-image model achieves a leading overall score of 0.86 on GenEval in system-level comparison. Codes are publicly available at https://github.com/Zehong-Ma/DeCo.