Paper 2607.19191
ABot-World-0: Infinite Interactive World Rollout on a Single Desktop GPU
- Published
- Jul 2026
- Research lab
- Independent
- Citations
- 0
- GitHub
- 1.8K stars
01 In brief
Summary
ABot-World-0 is an action-conditioned video world model for real-time, long-horizon closed-loop interaction, deployable on a single NVIDIA RTX 5090 GPU.
It uses raw keyboard inputs as a unified control interface for both scene roaming and third-person character control, with reference-character memory for identity consistency.
The model is trained on multi-source data (AAA games, simulations, internet videos) collected via WorldExplorer, an agent-driven system with training-feedback-driven rebalancing, and processed through a unified pipeline with 14 deterministic quality checks and VLM-based annotation.
The training pipeline progressively distills a bidirectional teacher into a causal student via teacher forcing and ODE distillation, and introduces LongForcing to align long student self-rollouts with an extended-horizon teacher, mitigating autoregressive drift.
For deployment, a co-designed streaming stack includes a lightweight VAE decoder, efficient attention, memory-aware scheduling, and low-bit DiT inference.
Optimized low-bit configurations achieve up to 16 FPS at 720P, 1.2 s action-to-first-frame latency, and ~19 GiB peak VRAM.
On WorldRoamBench, ABot-World-0 shows competitive controllability, visual quality, physics, and memory retention, and qualitative tests demonstrate hour- and day-scale rollouts with sustained coherence and plausible physical interactions.
The system is open-sourced at https://github.com/amap-cvlab/ABot-World.
Future work includes richer actions, multi-scale LongForcing, and more efficient decoding for broader hardware support.
The project was sponsored by Mu Xu and Ning Guo, with contributions from multiple teams at AMAP CV Lab, Alibaba Group.
02 From the paper
Abstract
We present ABot-World-0, an action-conditioned video world model for real-time, long-horizon closed-loop interaction, supported by a multi-source data infrastructure spanning AAA games, simulation engines, and internet videos to learn controllable world dynamics. WorldExplorer performs agent-driven collection guided by training feedback, while a unified pipeline applies 14 deterministic quality checks, VLM-based assessment, and synchronized action and text annotation. We progressively distill a bidirectional action-conditioned teacher into a causal student through teacher forcing and ODE distillation, and introduce LongForcing to align long student self-rollouts with an extended-horizon teacher, mitigating accumulated distribution shift and autoregressive drift. Raw keyboard actions provide a unified control interface for scene roaming and third-person character interaction, while reference-character memory provides persistent appearance cues for identity consistency during third-person rollouts. For deployment, we co-design a streaming inference stack with a lightweight VAE decoder, efficient attention, memory-aware scheduling, and low-bit DiT inference. Across optimized low-bit configurations, ABot-World-0 streams 720P video at up to 16 FPS on a single NVIDIA RTX 5090 desktop GPU, with 1.2s action-to-first-frame latency and approximately 19GiB peak VRAM. Experiments on WorldRoamBench and extended interactive rollouts demonstrate competitive controllability and coherent long-horizon world evolution.