Paper 2603.16669
Kinema4D: Kinematic 4D World Modeling for Spatiotemporal Embodied Simulation
- Published
- Mar 2026
- Research lab
- Independent
- Citations
- 6
- GitHub
- 83 stars
01 In brief
Summary
Kinema4D is a new action-conditioned 4D generative robotic simulator that models robot-world interactions as 4D spatiotemporal events.
It disentangles simulation into precise 4D robot control via URDF-based kinematics and generative 4D environmental reactions using a diffusion transformer that synthesizes synchronized RGB and pointmap sequences.
The authors curated Robo4D-200k, a dataset of 201,426 robot interaction episodes with 4D annotations, and demonstrate that Kinema4D outperforms existing 2D and 4D baselines on video and geometric metrics, including PSNR 22.50, SSIM 0.864, and F-Score 0.4733.
It also shows zero-shot out-of-distribution transfer capability in real-world policy evaluation, accurately simulating both successes and near-miss failures.
Ablations confirm the importance of pointmap control, 4D-aware joint modeling, and robustness to noise.
Limitations include occasional violations of physical laws due to statistical synthesis.
02 From the paper
Abstract
Simulating robot-world interactions is a cornerstone of Embodied AI. Recently, a few works have shown promise in leveraging video generations to transcend the rigid visual/physical constraints of traditional simulators. However, they primarily operate in 2D space or are guided by static environmental cues, ignoring the fundamental reality that robot-world interactions are inherently 4D spatiotemporal events that require precise interactive modeling. To restore this 4D essence while ensuring the precise robot control, we introduce Kinema4D, a new action-conditioned 4D generative robotic simulator that disentangles the robot-world interaction into: i) Precise 4D representation of robot controls: we drive a URDF-based 3D robot via kinematics, producing a precise 4D robot control trajectory. ii) Generative 4D modeling of environmental reactions: we project the 4D robot trajectory into a pointmap as a spatiotemporal visual signal, controlling the generative model to synthesize complex environments' reactive dynamics into synchronized RGB/pointmap sequences. To facilitate training, we curated a large-scale dataset called Robo4D-200k, comprising 201,426 robot interaction episodes with high-quality 4D annotations. Extensive experiments demonstrate that our method effectively simulates physically-plausible, geometry-consistent, and embodiment-agnostic interactions that faithfully mirror diverse real-world dynamics. For the first time, it shows potential zero-shot transfer capability, providing a high-fidelity foundation for advancing next-generation embodied simulation.