Paper 2509.04338
From Editor to Dense Geometry Estimator
- Published
- Sep 2025
- Research lab
- Independent
- Citations
- 25
- GitHub
- 243 stars
01 In brief
Summary
FE2E is a framework that adapts a pre-trained image editing model, Step1X-Edit, for monocular dense geometry prediction (depth and normal estimation).
The authors argue that editing models, unlike text-to-image generators, possess inherent structural priors that make them more suitable for image-to-image tasks.
They introduce three key adaptations: a consistent velocity flow matching objective with a fixed starting point for stable training, logarithmic quantization of depth annotations to resolve precision issues with BF16, and a cost-free joint estimation of depth and normals by repurposing the discarded output region of the DiT model.
Trained on only 71K images, FE2E achieves state-of-the-art zero-shot performance, with over 35% improvement on ETH3D and outperforming DepthAnything, which uses 100x more data.
Ablations confirm the effectiveness of each component and the generalizability to other editors like FLUX-Kontext.
02 From the paper
Abstract
Leveraging visual priors from pre-trained text-to-image (T2I) generative models has shown success in dense prediction. However, dense prediction is inherently an image-to-image task, suggesting that image editing models, rather than T2I generative models, may be a more suitable foundation for fine-tuning. Motivated by this, we conduct a systematic analysis of the fine-tuning behaviors of both editors and generators for dense geometry estimation. Our findings show that editing models possess inherent structural priors, which enable them to converge more stably by ``refining" their innate features, and ultimately achieve higher performance than their generative counterparts. Based on these findings, we introduce \textbf{FE2E}, a framework that pioneeringly adapts an advanced editing model based on Diffusion Transformer (DiT) architecture for dense geometry prediction. Specifically, to tailor the editor for this deterministic task, we reformulate the editor's original flow matching loss into the ``consistent velocity" training objective. And we use logarithmic quantization to resolve the precision conflict between the editor's native BFloat16 format and the high precision demand of our tasks. Additionally, we leverage the DiT's global attention for a cost-free joint estimation of depth and normals in a single forward pass, enabling their supervisory signals to mutually enhance each other. Without scaling up the training data, FE2E achieves impressive performance improvements in zero-shot monocular depth and normal estimation across multiple datasets. Notably, it achieves over 35\% performance gains on the ETH3D dataset and outperforms the DepthAnything series, which is trained on 100$\times$ data. The project page can be accessed \href{https://amap-ml.github.io/FE2E/}{here}.