Paper 2603.21986
Speed by Simplicity: A Single-Stream Architecture for Fast Audio-Video Generative Foundation Model
- Published
- Mar 2026
- Research lab
- Independent
- Citations
- 12
- GitHub
- 2.1K stars
01 In brief
Summary
daVinci-MagiHuman is an open-source audio-video generative foundation model for human-centric generation, jointly producing synchronized video and audio via a single-stream Transformer that processes text, video, and audio in a unified token sequence using self-attention only.
This design avoids multi-stream complexity and supports multilingual generation (Chinese, English, Japanese, Korean, German, French).
The model uses a 15B-parameter, 40-layer backbone with a sandwich layout, timestep-free denoising, per-head gating, and unified conditioning.
Inference is accelerated via latent-space super-resolution, a Turbo VAE decoder, full-graph compilation, and DMD-2 distillation, enabling 5-second 256p video generation in 2 seconds on a single H100 GPU.
In automatic evaluation, it achieves the highest visual quality and text alignment among open models, with the lowest word error rate (14.60%).
In pairwise human evaluation, it wins 80.0% against Ovi 1.1 and 60.9% against LTX 2.3 over 2,000 comparisons.
The complete model stack, including base, distilled, and super-resolution models, is open-sourced.
The model is particularly strong in expressive facial performance, speech-expression coordination, body motion, and audio-video synchronization, making it suitable for both offline content creation and latency-sensitive interactive applications.
The architecture is hardware-friendly and infrastructure-friendly, simplifying optimization and future research.
The release aims to provide a practical and extensible foundation for the community.
The model's design choices, including the sandwich layout and timestep-free denoising, contribute to its simplicity and effectiveness.
The super-resolution stage refines video in latent space, preserving audio-video coupling.
The Turbo VAE decoder reduces decoding overhead, and full-graph compilation provides a 1.2× speedup.
The distilled model uses only 8 denoising steps…
02 From the paper
Abstract
We present daVinci-MagiHuman, an open-source audio-video generative foundation model for human-centric generation. daVinci-MagiHuman jointly generates synchronized video and audio using a single-stream Transformer that processes text, video, and audio within a unified token sequence via self-attention only. This single-stream design avoids the complexity of multi-stream or cross-attention architectures while remaining easy to optimize with standard training and inference infrastructure. The model is particularly strong in human-centric scenarios, producing expressive facial performance, natural speech-expression coordination, realistic body motion, and precise audio-video synchronization. It supports multilingual spoken generation across Chinese (Mandarin and Cantonese), English, Japanese, Korean, German, and French. For efficient inference, we combine the single-stream backbone with model distillation, latent-space super-resolution, and a Turbo VAE decoder, enabling generation of a 5-second 256p video in 2 seconds on a single H100 GPU. In automatic evaluation, daVinci-MagiHuman achieves the highest visual quality and text alignment among leading open models, along with the lowest word error rate (14.60%) for speech intelligibility. In pairwise human evaluation, it achieves win rates of 80.0% against Ovi 1.1 and 60.9% against LTX 2.3 over 2000 comparisons. We open-source the complete model stack, including the base model, the distilled model, the super-resolution model, and the inference codebase.