Paper 2511.13648
PhysX-Anything: Simulation-Ready Physical 3D Assets from Single Image
- Published
- Nov 2025
- Research lab
- Independent
- Citations
- 25
- GitHub
- 914 stars
01 In brief
Summary
PhysX-Anything is a novel framework that generates simulation-ready physical 3D assets from a single in-the-wild image.
It uses a VLM-based generative model to predict geometry, articulation, and physical properties, and introduces a voxel-based representation that reduces token count by 193x, enabling explicit geometry learning without special tokens.
The framework includes a controllable flow transformer for fine-grained geometry and outputs URDF/XML files for direct use in simulators.
To address dataset scarcity, the authors construct PhysX-Mobility, a dataset with 47 categories and over 2K objects, expanding prior physical 3D datasets by over 2x.
Experiments show PhysX-Anything outperforms state-of-the-art methods (URDFormer, Articulate-Anything, PhysXGen) on geometry and physical metrics, with a 99% improvement in absolute scale error.
In-the-wild evaluations and user studies confirm robust generalization.
Simulation-based tests in a MuJoCo-style environment demonstrate the assets' utility for contact-rich robotic policy learning, such as manipulating eyeglasses.
The work aims to bridge the gap between synthetic 3D assets and real-world applications in embodied AI and robotics.
02 From the paper
Abstract
3D modeling is shifting from static visual representations toward physical, articulated assets that can be directly used in simulation and interaction. However, most existing 3D generation methods overlook key physical and articulation properties, thereby limiting their utility in embodied AI. To bridge this gap, we introduce PhysX-Anything, the first simulation-ready physical 3D generative framework that, given a single in-the-wild image, produces high-quality sim-ready 3D assets with explicit geometry, articulation, and physical attributes. Specifically, we propose the first VLM-based physical 3D generative model, along with a new 3D representation that efficiently tokenizes geometry. It reduces the number of tokens by 193x, enabling explicit geometry learning within standard VLM token budgets without introducing any special tokens during fine-tuning and significantly improving generative quality. In addition, to overcome the limited diversity of existing physical 3D datasets, we construct a new dataset, PhysX-Mobility, which expands the object categories in prior physical 3D datasets by over 2x and includes more than 2K common real-world objects with rich physical annotations. Extensive experiments on PhysX-Mobility and in-the-wild images demonstrate that PhysX-Anything delivers strong generative performance and robust generalization. Furthermore, simulation-based experiments in a MuJoCo-style environment validate that our sim-ready assets can be directly used for contact-rich robotic policy learning. We believe PhysX-Anything can substantially empower a broad range of downstream applications, especially in embodied AI and physics-based simulation.