Paper 2601.03233
LTX-2: Efficient Joint Audio-Visual Foundation Model
- Published
- Jan 2026
- Research lab
- Independent
- Citations
- 135
- GitHub
- 8.5K stars
01 In brief
Summary
LTX-2 is an open-source text-to-audio+video (T2AV) foundation model that jointly generates synchronized video and audio from text.
It uses an asymmetric dual-stream transformer with a 14B-parameter video stream and a 5B-parameter audio stream, connected via bidirectional cross-attention layers with temporal positional embeddings and cross-modality AdaLN.
The model employs separate modality-specific VAEs, a multilingual text encoder (Gemma3-12B) with multi-layer feature extraction and thinking tokens, and a modality-aware classifier-free guidance (modality-CFG) mechanism.
LTX-2 generates speech, foley, and ambient sound synchronized with visual content.
Evaluations show it outperforms open-source systems like Ovi and matches proprietary models like Veo 3 and Sora 2 in audiovisual quality, while being about 18× faster than Wan 2.2-14B (1.22s vs 22.30s per step on H100).
It supports up to 20 seconds of continuous video with stereo audio.
Limitations include performance variability across languages, occasional speaker confusion in multi-speaker scenarios, temporal drift beyond 20 seconds, and lack of explicit reasoning.
All model weights and code are publicly released.
02 From the paper
Abstract
Recent text-to-video diffusion models can generate compelling video sequences, yet they remain silent -- missing the semantic, emotional, and atmospheric cues that audio provides. We introduce LTX-2, an open-source foundational model capable of generating high-quality, temporally synchronized audiovisual content in a unified manner. LTX-2 consists of an asymmetric dual-stream transformer with a 14B-parameter video stream and a 5B-parameter audio stream, coupled through bidirectional audio-video cross-attention layers with temporal positional embeddings and cross-modality AdaLN for shared timestep conditioning. This architecture enables efficient training and inference of a unified audiovisual model while allocating more capacity for video generation than audio generation. We employ a multilingual text encoder for broader prompt understanding and introduce a modality-aware classifier-free guidance (modality-CFG) mechanism for improved audiovisual alignment and controllability. Beyond generating speech, LTX-2 produces rich, coherent audio tracks that follow the characters, environment, style, and emotion of each scene -- complete with natural background and foley elements. In our evaluations, the model achieves state-of-the-art audiovisual quality and prompt adherence among open-source systems, while delivering results comparable to proprietary models at a fraction of their computational cost and inference time. All model weights and code are publicly released.