Paper 2607.05373
PixWorld: Unifying 3D Scene Generation and Reconstruction in Pixel Space
- Published
- Jul 2026
- Research lab
- Independent
- Citations
- 1
- GitHub
- 239 stars
01 In brief
Summary
PixWorld is a unified pixel-space diffusion framework for 3D scene generation and reconstruction.
It partitions multi-view inputs into clean and noisy subsets, processes them with a two-stream diffusion transformer, and decodes features into a pixel-aligned 3D Gaussian representation.
The diffusion objective is applied directly on rendered images, eliminating the need for a pretrained VAE or RAE and avoiding information loss.
A geometry perception loss aligns rendered views with ground truth in the feature space of a frozen 3D foundation model (π³), providing 3D structural supervision.
Experiments on RealEstate10K, DL3DV-10K, and WorldScore show PixWorld outperforms prior latent-space generation methods and matches state-of-the-art reconstruction methods.
For example, it achieves the best PSNR and LPIPS on reconstruction benchmarks, and on single-image generation it improves PSNR by +1.06 dB on RealEstate10K and +0.75 dB on DL3DV-10K over the best baseline.
Ablations confirm the geometry perception loss is crucial, improving PSNR by 1.13 dB and AUC@5 by 12.5% relative.
The model has 1.04B parameters and generates a scene in 15 seconds.
02 From the paper
Abstract
3D reconstruction and generation are commonly tackled by separate paradigms: pixel-based regression for reconstruction, and latent diffusion for generation. Recent works attempt to unify them in latent space, but with notable drawbacks: the diffusion objective is defined on latent features rather than the underlying 3D representation, and both branches suffer from information loss introduced by latent encoding, while requiring a pretrained Variational Autoencoder (VAE) or Representation Autoencoder (RAE). In this paper, we reformulate these two tasks under a unified pixel-space diffusion paradigm and introduce PixWorld, a single model that jointly addresses 3D reconstruction and generation. By supervising diffusion directly on rendered images, PixWorld removes the above limitations and aligns optimization with 3D scene fidelity. Beyond photometric and perceptual supervision that operates at the 2D image level and lacks 3D geometric awareness, we further introduce a geometry perception loss that aligns rendered views with their ground truth in the geometry-aware feature space of a pretrained 3D foundation model, providing 3D structural supervision. PixWorld consistently outperforms prior latent-space generation methods and matches state-of-the-art reconstruction methods, demonstrating the superiority of a unified pixel-space approach.