Paper 2512.04677
Live Avatar: Streaming Real-time Audio-Driven Avatar Generation with Infinite Length
- Published
- Dec 2025
- Research lab
- Independent
- Citations
- 0
- GitHub
- 2.3K stars
01 In brief
Summary
Live Avatar is an algorithm-system co-designed framework enabling real-time, streaming, and infinite-length audio-driven avatar generation using a 14-billion-parameter diffusion model.
It addresses two key challenges: long-horizon consistency and the real-time-fidelity trade-off.
The algorithm side uses a two-stage pipeline (Diffusion Forcing pretraining with motion-frame scaffolding, then Self-Forcing distillation) to convert a bidirectional model into a causal, few-step streaming one.
It stores only noisy representations in the KV cache (History Corrupt), which suppresses error accumulation and enables stable generation beyond 10,000 seconds.
Additional strategies include Adaptive Attention Sink (AAS) and Rolling RoPE to mitigate distribution drift and positional extrapolation.
The system side introduces Timestep-forcing Pipeline Parallelism (TPP), assigning each GPU a fixed denoising timestep to break the sequential sampling bottleneck, achieving 45 FPS with a TTFF of 1.21 s on 5 H100 GPUs.
The framework also introduces GenBench, a benchmark with long-video test cases exceeding five minutes.
Experiments show superior performance over existing methods in quality, identity consistency, and speed, with ablations confirming the effectiveness of each component.
The main limitation is an end-to-end latency of about 3 seconds, not yet meeting seamless bidirectional interaction requirements.
The project page is at https://liveavatar.github.io/.
02 From the paper
Abstract
Audio-driven avatar interaction demands real-time, streaming, and infinite-length generation -- capabilities fundamentally at odds with the sequential denoising and long-horizon drift of current diffusion models. We present Live Avatar, an algorithm-system co-designed framework that addresses both challenges for a 14-billion-parameter diffusion model. On the algorithm side, a two-stage pipeline distills a pretrained bidirectional model into a causal, few-step streaming one, while a set of complementary long-horizon strategies eliminate identity drift and visual artifacts, enabling stable autoregressive generation exceeding 10000 seconds. On the system side, Timestep-forcing Pipeline Parallelism (TPP) assigns each GPU a fixed denoising timestep, converting the sequential diffusion chain into an asynchronous spatial pipeline that simultaneously boosts throughput and improves temporal consistency. Live Avatar achieves 45 FPS with a TTFF of 1.21\,s on 5 H800 GPUs, and to our knowledge is the first to enable practical real-time streaming of a 14B diffusion model for infinite-length avatar generation. We further introduce GenBench, a standardized long-form benchmark, to facilitate reproducible evaluation. Our project page is at https://liveavatar.github.io/.