Paper 2603.16871
WorldCam: Interactive Autoregressive 3D Gaming Worlds with Camera Pose as a Unifying Geometric Representation
- Published
- Mar 2026
- Research lab
- Independent
- Citations
- 6
- GitHub
- 179 stars
01 In brief
Summary
WorldCam is an interactive 3D gaming world model that uses camera pose as a unifying geometric representation to achieve precise action control, long-horizon generation, and 3D consistency.
It maps user actions (keyboard/mouse) to 6-DoF camera poses via Lie algebra, injecting them into a video DiT (Wan-2.1) for accurate control.
Global camera poses serve as spatial indices for retrieving relevant past latents from a long-term memory pool, ensuring geometric consistency when revisiting locations.
The model employs progressive noise scheduling, attention sinks, and short-term memory for stable long-horizon inference.
The authors introduce WorldCam-50h, a dataset of 3,000 minutes of human gameplay from Counter-Strike, Xonotic, and Unvanquished, annotated with camera poses and text captions.
Experiments show WorldCam outperforms prior interactive world models (Yume, Matrix-Game 2.0, GameCraft) and camera-controlled methods (CameraCtrl, MotionCtrl) in action controllability, visual quality, and 3D consistency, with improvements like 16.3% lower RPE_camera vs.
GameCraft and 8.1% higher VBench average score.
Ablations validate the Lie algebra mapping, pose-based memory retrieval, and memory sizes.
02 From the paper
Abstract
Recent advances in video diffusion transformers have enabled interactive gaming world models that allow users to explore generated environments over extended horizons. However, existing approaches struggle with precise action control and long-horizon 3D consistency. Most prior works treat user actions as abstract conditioning signals, overlooking the fundamental geometric coupling between actions and the 3D world, whereby actions induce relative camera motions that accumulate into a global camera pose within a 3D world. In this paper, we establish camera pose as a unifying geometric representation to jointly ground immediate action control and long-term 3D consistency. First, we define a physics-based continuous action space and represent user inputs in the Lie algebra to derive precise 6-DoF camera poses, which are injected into the generative model via a camera embedder to ensure accurate action alignment. Second, we use global camera poses as spatial indices to retrieve relevant past observations, enabling geometrically consistent revisiting of locations during long-horizon navigation. To support this research, we introduce a large-scale dataset comprising 3,000 minutes of authentic human gameplay annotated with camera trajectories and textual descriptions. Extensive experiments show that our approach substantially outperforms state-of-the-art interactive gaming world models in action controllability, long-horizon visual quality, and 3D spatial consistency.