Paper 2603.25746
ShotStream: Streaming Multi-Shot Video Generation for Interactive Storytelling
- Published
- Mar 2026
- Research lab
- Independent
- Citations
- 14
- GitHub
- 177 stars
01 In brief
Summary
ShotStream is a novel causal multi-shot video generation architecture that enables interactive storytelling and real-time synthesis at 16 FPS on a single GPU.
It reformulates multi-shot generation as a next-shot prediction task, allowing users to guide narratives via streaming prompts.
The method first fine-tunes a text-to-video model into a bidirectional next-shot teacher, then distills it into a 4-step causal student using Distribution Matching Distillation.
To maintain inter-shot and intra-shot consistency, a dual-cache memory mechanism is introduced, with a RoPE discontinuity indicator distinguishing global and local caches.
A two-stage distillation strategy (intra-shot then inter-shot self-forcing) mitigates error accumulation.
Experiments show ShotStream outperforms bidirectional and autoregressive baselines in consistency, prompt adherence, and transition control, with over 25x throughput improvement over bidirectional models.
User studies confirm preference for ShotStream in visual consistency, quality, and prompt following.
Ablations validate the dynamic context sampling, multi-caption injection, frame concatenation, RoPE offset, and two-stage training.
Limitations include artifacts in highly complex scenes and potential for further acceleration.
02 From the paper
Abstract
Multi-shot video generation is crucial for long narrative storytelling, yet current bidirectional architectures suffer from limited interactivity and high latency. We propose ShotStream, a novel causal multi-shot architecture that enables interactive storytelling and efficient on-the-fly frame generation. By reformulating the task as next-shot generation conditioned on historical context, ShotStream allows users to dynamically instruct ongoing narratives via streaming prompts. We achieve this by first fine-tuning a text-to-video model into a bidirectional next-shot generator, which is then distilled into a causal student via Distribution Matching Distillation. To overcome the challenges of inter-shot consistency and error accumulation inherent in autoregressive generation, we introduce two key innovations. First, a dual-cache memory mechanism preserves visual coherence: a global context cache retains conditional frames for inter-shot consistency, while a local context cache holds generated frames within the current shot for intra-shot consistency. And a RoPE discontinuity indicator is employed to explicitly distinguish the two caches to eliminate ambiguity. Second, to mitigate error accumulation, we propose a two-stage distillation strategy. This begins with intra-shot self-forcing conditioned on ground-truth historical shots and progressively extends to inter-shot self-forcing using self-generated histories, effectively bridging the train-test gap. Extensive experiments demonstrate that ShotStream generates coherent multi-shot videos with sub-second latency, achieving 16 FPS on a single GPU. It matches or exceeds the quality of slower bidirectional models, paving the way for real-time interactive storytelling. Training and inference code, as well as the models, are available on our