The Year in AI Papers
The year/Meta AI

Paper 2511.16624

SAM 3D: 3Dfy Anything in Images

SAM 3D Team, Xingyu Chen, Fu-Jen Chu, Pierre Gleize, Kevin J Liang, Alexander Sax, Hao Tang, Weiyao Wang, Michelle Guo, Thibaut Hardin, Xiang Li, Aohan Lin, et al.

Published
Nov 2025
Research lab
Meta AI
Citations
197
GitHub
7.2K stars

01 In brief

Summary

SAM 3D is a generative model for 3D object reconstruction from a single image, predicting geometry, texture, and layout.

It excels in natural images with occlusion and clutter, using a human- and model-in-the-loop pipeline to create large-scale 3D annotation data.

The model uses a multi-stage training framework: synthetic pretraining on 2.7M meshes (Iso-3DO), mid-training with render-paste data (RP-3DO), and post-training with real-world alignment via SFT and DPO.

It achieves significant gains, with at least a 5:1 win rate in human preference tests.

The authors release code, model weights, a demo, and a new benchmark (SA-3DAO) with 1,000 artist-created meshes.

Key results include F1@0.01 of 0.2344 on SA-3DAO, outperforming baselines like Trellis (0.1475).

The model also shows strong layout estimation, with ADD-S@0.1 of 0.7232 on SA-3DAO.

The data engine iteratively improves performance, and the model supports optional pointmap conditioning.

Limitations include resolution constraints and lack of multi-object reasoning.

02 From the paper

Abstract

We present SAM 3D, a generative model for visually grounded 3D object reconstruction, predicting geometry, texture, and layout from a single image. SAM 3D excels in natural images, where occlusion and scene clutter are common and visual recognition cues from context play a larger role. We achieve this with a human- and model-in-the-loop pipeline for annotating object shape, texture, and pose, providing visually grounded 3D reconstruction data at unprecedented scale. We learn from this data in a modern, multi-stage training framework that combines synthetic pretraining with real-world alignment, breaking the 3D "data barrier". We obtain significant gains over recent work, with at least a 5:1 win rate in human preference tests on real-world objects and scenes. We will release our code and model weights, an online demo, and a new challenging benchmark for in-the-wild 3D object reconstruction.