Paper 2511.21631
Qwen3-VL Technical Report
01 In brief
Summary
Qwen3-VL is a state-of-the-art vision-language model family from the Qwen team, released on December 1, 2025.
It supports interleaved contexts up to 256K tokens and comes in dense (2B/4B/8B/32B) and MoE (30B-A3B/235B-A22B) variants.
Key architectural innovations include interleaved-MRoPE for balanced spatial-temporal encoding, DeepStack for multi-level ViT feature fusion, and text-based timestamps for video temporal grounding.
Training uses square-root reweighting to balance text and multimodal objectives, with pretraining extended to 256K tokens and post-training split into non-thinking and thinking variants.
The model achieves superior performance on benchmarks like MMMU, MathVista, and MathVision, and excels in long-context understanding, OCR, grounding, and video tasks.
It also maintains strong pure-text capabilities, often surpassing text-only backbones.
The model is released under Apache 2.0 license, with resources available on Hugging Face, ModelScope, and GitHub.
The report details extensive evaluations across general VQA, multimodal reasoning, document understanding, grounding, video, and agent tasks, demonstrating competitive or superior results against leading models like Gemini 2.5 Pro and GPT-5.
The flagship 235B-A22B model achieves state-of-the-art on many benchmarks, and smaller variants also show strong performance, highlighting scalability and efficiency.
The model is positioned as a foundation for image-grounded reasoning, agentic decision-making, and multimodal code intelligence.
Future work aims at embodied AI agents and unified understanding-generation architectures.
The report includes detailed ablation studies confirming the effectiveness of architectural choices, such as DeepStack and the vision encoder improvements.
The model's long-context capability is validated by a needle-in-a-haystack test showing 100% accuracy up to 30 minutes of video and 99.5% at 1M tokens…
02 From the paper
Abstract
We introduce Qwen3-VL, the most capable vision-language model in the Qwen series to date, achieving superior performance across a broad range of multimodal benchmarks. It natively supports interleaved contexts of up to 256K tokens, seamlessly integrating text, images, and video. The model family includes both dense (2B/4B/8B/32B) and mixture-of-experts (30B-A3B/235B-A22B) variants to accommodate diverse latency-quality trade-offs. Qwen3-VL delivers three core pillars: (i) markedly stronger pure-text understanding, surpassing comparable text-only backbones in several cases; (ii) robust long-context comprehension with a native 256K-token window for both text and interleaved multimodal inputs, enabling faithful retention, retrieval, and cross-referencing across long documents and videos; and (iii) advanced multimodal reasoning across single-image, multi-image, and video tasks, demonstrating leading performance on comprehensive evaluations such as MMMU and visual-math benchmarks (e.g., MathVista and MathVision). Architecturally, we introduce three key upgrades: (i) an enhanced interleaved-MRoPE for stronger spatial-temporal modeling across images and video; (ii) DeepStack integration, which effectively leverages multi-level ViT features to tighten vision-language alignment; and (iii) text-based time alignment for video, evolving from T-RoPE to explicit textual timestamp alignment for more precise temporal grounding. Under comparable token budgets and latency constraints, Qwen3-VL achieves superior performance in both dense and Mixture-of-Experts (MoE) architectures. We envision Qwen3-VL serving as a foundational engine for image-grounded reasoning, agentic decision-making, and multimodal code intelligence in real-world workflows.