Paper 2512.22096
Yume-1.5: A Text-Controlled Interactive World Generation Model
- Published
- Dec 2025
- Research lab
- Independent
- Citations
- 52
- GitHub
- 681 stars
01 In brief
Summary
Yume1.5 is a framework for generating interactive, continuous virtual worlds from a single image or text prompt, with keyboard-based control for person and camera movement.
It addresses limitations in existing video diffusion models, such as limited generalizability, high latency, and insufficient text control.
The framework introduces three core innovations: (1) Joint Temporal-Spatial-Channel Modeling (TSCM) for efficient long-video generation, which compresses historical frames along temporal-spatial and channel dimensions to maintain stable sampling speed; (2) a real-time acceleration strategy combining Self-Forcing with TSCM to reduce error accumulation and enable few-step inference; and (3) text-controlled world event generation, achieved through architectural design and mixed-dataset training.
The model is trained on real-world, synthetic, and event datasets, and uses a decomposed text encoding scheme for event and action descriptions.
In experiments, Yume1.5 achieves an instruction-following score of 0.836, outperforming baselines, and maintains stable quality in long-video generation.
It generates 12 fps at 540p on a single A100 GPU.
Limitations include artifacts like vehicles moving backwards and performance degradation in high crowd density, with future work exploring Mixture-of-Experts architectures.
02 From the paper
Abstract
Recent approaches have demonstrated the promise of using diffusion models to generate interactive and explorable worlds. However, most of these methods face critical challenges such as excessively large parameter sizes, reliance on lengthy inference steps, and rapidly growing historical context, which severely limit real-time performance and lack text-controlled generation capabilities. To address these challenges, we propose \method, a novel framework designed to generate realistic, interactive, and continuous worlds from a single image or text prompt. \method achieves this through a carefully designed framework that supports keyboard-based exploration of the generated worlds. The framework comprises three core components: (1) a long-video generation framework integrating unified context compression with linear attention; (2) a real-time streaming acceleration strategy powered by bidirectional attention distillation and an enhanced text embedding scheme; (3) a text-controlled method for generating world events. We have provided the codebase in the supplementary material.