Paper 2606.26740
LiveEdit: Towards Real-Time Diffusion-Based Streaming Video Editing
- Published
- Jun 2026
- Research lab
- Independent
- Citations
- 2
- GitHub
- 147 stars
01 In brief
Summary
LiveEdit is a novel streaming video editing framework that performs causal, chunk-by-chunk editing with high fidelity and ultra-low latency.
It addresses two core issues: attention distribution shift and spatial-temporal token redundancy.
The method uses a three-stage distillation pipeline: Stage 1 tunes a bidirectional DiT for editing, Stage 2 transitions to a causal DiT via teacher forcing, and Stage 3 applies DMD to compress inference to 4 steps.
An AR-oriented Mask Cache reuses features in unedited regions, reducing computation.
The framework achieves 12.66 FPS and 79ms per-frame latency, outperforming baselines in text alignment, background consistency, and motion smoothness.
A user study confirms its superiority in instruction consistency, background preservation, and overall quality.
02 From the paper
Abstract
Streaming video editing has made rapid progress, yet practical deployment is still limited by two core issues: maintaining stable backgrounds and non-edited regions over time, and achieving the low latency required for real-time interactive scenarios. Meanwhile, recent streaming video generation methods are mostly developed for synthesis and cannot be directly applied to editing due to the strict preservation requirement and region-specific control. In this work, we present a novel streaming video editing framework that performs causal, frame-by-frame editing with strong content preservation and real-time responsiveness. Our key design is a three-stage distillation pipeline that progressively transfers editing capability from a powerful bidirectional foundation model to an efficient unidirectional streaming editor, enabling stable long-horizon edits without sacrificing visual fidelity. To further support real-time deployment, we introduce an AR-oriented mask cache that reuses region-related computation across frames, substantially reducing redundant processing and accelerating inference. Finally, we establish a dedicated benchmark for streaming video editing. Extensive evaluations demonstrate that our method achieves state-of-the-art visual quality among streaming baselines while drastically boosting inference speed to 12.66 FPS, making it suitable for interactive and augmented reality applications.