Qwen3-Omni Technical Report

Qwen3-Omni is a single multimodal model that achieves state-of-the-art performance across text, image, audio, and video without degradation relative to single-modal counterparts.

It matches the performance of same-sized Qwen single-modal models and excels on audio tasks, achieving open-source SOTA on 32 of 36 audio benchmarks and overall SOTA on 22, outperforming closed-source models like Gemini-2.5-Pro and GPT-4o-Transcribe.

The model uses a Thinker–Talker Mixture-of-Experts (MoE) architecture, with the Thinker handling text generation and the Talker generating streaming speech tokens.

It supports text in 119 languages, speech understanding in 19, and speech generation in 10, and can process audio recordings up to 40 minutes.

The Talker uses a multi-codebook scheme with a lightweight causal ConvNet for streaming synthesis, achieving a theoretical end-to-end first-packet latency of 234 ms.

The model also includes a Thinking variant for multimodal reasoning and a Captioner for detailed audio captions.

Qwen3-Omni-30B-A3B, its Thinking variant, and Captioner are released under Apache 2.0.

Key improvements over Qwen2.5-Omni include a new AuT audio encoder, multi-codebook speech generation, and support for long audio inputs and more languages.

The model demonstrates non-degrading multimodal performance, with early integration of multimodal data during pretraining enhancing cross-modal capabilities without harming text or vision performance.

Evaluations show strong results in ASR, speech translation, voice interaction, audio reasoning, music understanding, vision tasks, and audiovisual reasoning.

Speech generation evaluations show competitive or superior performance in zero-shot, multilingual, and cross-lingual scenarios.

The model's architecture includes chunked prefilling and MoE for high concurrency, with latency and throughput metrics provided for various…