Paper 2607.03118
Vidu S1: A Real-Time Interactive Video Generation Model
- Published
- Jul 2026
- Research lab
- Independent
- Citations
- 2
- GitHub
- 240 stars
01 In brief
Summary
Vidu S1 is a real-time interactive video generation model that enables users to control digital characters via voice instructions during generation, supporting infinite-length video without blurring or drift.
Built with TurboDiffusion and TurboServe, it outputs 540p video at up to 42 FPS on consumer GPUs.
The model uses a three-stage training pipeline: bidirectional teacher training, causal teacher training with hybrid Teacher Forcing and Diffusion Forcing, and DMD distillation with PCM regularization.
Inference employs sliding-window decoding with RoPE repositioning and TwinCache for efficiency.
On Vidu-StreamBench, Vidu S1 achieves 100% preference in subject controllability against HeyGen and LemonSlice, and on HDTF it leads with CSIM 0.9192, Sync-D 7.847, and DOVER 0.5660, outperforming commercial systems like HeyGen, Kling Avatar 2.0, and open-source models.
The model supports custom character images and various voice tones, making it suitable for live conversation, virtual hosts, and educational agents.
02 From the paper
Abstract
We introduce Vidu S1, a real-time interactive video generation model supporting voice control of digital characters. Users can control video generation content at any moment through voice instructions. Vidu S1 supports infinite-length real-time video generation without blurring, drift, or visual distortion. Built with TurboDiffusion and TurboServe, Vidu S1 outputs 540p real-time videos at up to 42 FPS on regular consumer GPUs. Users can upload custom images of real people, anime, and pets, and choose different voice tones for personalized experiences. Experiments show that Vidu S1 achieves the best performance across all test metrics while fully meeting real-time inference requirements. A playable online demo is available at https://vidu.com/vidu-stream.