Paper 2510.08377
UniVideo: Unified Understanding, Generation, and Editing for Videos
- Published
- Oct 2025
- Research lab
- Independent
- Citations
- 68
- GitHub
- 546 stars
01 In brief
Summary
UniVideo is a unified multimodal framework for video understanding, generation, and editing, presented at ICLR 2026.
It uses a dual-stream architecture combining a frozen Multimodal Large Language Model (MLLM, Qwen2.5VL-7B) for understanding and a Multimodal DiT (MMDiT, HunyuanVideo-T2V-13B) for generation, connected via a trainable MLP connector.
This design preserves text generation, interprets complex multimodal instructions, and maintains visual consistency.
UniVideo is jointly trained across text-to-image, text-to-video, image-to-video, in-context video generation, in-context video editing, and image editing tasks in three stages.
Experiments show competitive or superior performance versus task-specific baselines on benchmarks like VBench, GenEval, ImgEdit-Bench, and UNICBench.
Notably, UniVideo generalizes zero-shot to unseen free-form video editing (e.g., changing materials or weather) and novel task compositions (e.g., combining editing with style transfer), and supports generation with thinking by interpreting visual prompts.
Ablations confirm multi-task learning and feeding visual inputs to both branches improve identity preservation and performance.
02 From the paper
Abstract
Unified multimodal models have shown promising results in multimodal content generation and editing but remain largely limited to the image domain. In this work, we present UniVideo, a versatile framework that extends unified modeling to the video domain. UniVideo adopts a dual-stream design, combining a Multimodal Large Language Model (MLLM) for instruction understanding with a Multimodal DiT (MMDiT) for video generation. This design preserves the MLLM's original text generation capabilities, enables accurate interpretation of complex multimodal instructions, and maintains visual consistency in the generated content. Built on this architecture, UniVideo unifies diverse video generation and editing tasks under a single multimodal instruction paradigm and is jointly trained across them. Extensive experiments demonstrate that UniVideo matches or surpasses state-of-the-art task-specific baselines in text/image-to-video generation, in-context video generation and in-context video editing. Notably, the unified design of UniVideo enables two forms of generalization. First, UniVideo supports task composition, such as combining editing with style transfer, by integrating multiple capabilities within a single instruction. Second, even without explicit training on free-form video editing, UniVideo transfers its editing capability from large-scale image editing data to this setting, handling unseen instructions such as changing the environment or altering materials within a video. Beyond these core capabilities, UniVideo also supports visual-prompt-based video generation, where the MLLM interprets visual prompts and guides the MMDiT during synthesis. To foster future research, we released our model and code.