The Year in AI Papers
The year/Independent research

Paper 2602.21818

SkyReels-V4: Multi-modal Video-Audio Generation, Inpainting and Editing model

Guibin Chen, Dixuan Lin, Jiangping Yang, Youqiang Zhang, Zhengcong Fei, Debang Li, Sheng Chen, Chaofeng Ao, Nuo Pang, Yiming Wang, Yikun Dou, Zheng Chen, et al.

Published
Feb 2026
Research lab
Independent
Citations
6
GitHub
Not linked

01 In brief

Summary

SkyReels-V4 is a unified multi-modal video foundation model for joint video-audio generation, inpainting, and editing.

It uses a dual-stream Multimodal Diffusion Transformer (MMDiT) architecture with separate video and audio branches sharing a Multimodal Large Language Model (MLLM) text encoder.

The model accepts text, images, video clips, masks, and audio references.

Video tasks are unified via channel concatenation as inpainting, and multi-modal references are injected via in-context learning.

It supports up to 1080p, 32 FPS, and 15-second durations.

An efficiency strategy generates low-resolution full sequences and high-resolution keyframes, followed by super-resolution and frame interpolation.

Training is progressive: video pretraining (T2I to multi-modal conditioning), audio pretraining, joint video-audio training, and supervised fine-tuning.

On the Artificial Analysis Arena, it ranks #1 in text-to-video with audio and #2 without audio.

In human evaluations (SkyReels-VABench), it outperforms baselines like Veo 3.1, Kling 2.6, Seedance 1.5 Pro, and Wan 2.6, especially in instruction following and motion quality.

02 From the paper

Abstract

SkyReels V4 is a unified multi modal video foundation model for joint video audio generation, inpainting, and editing. The model adopts a dual stream Multimodal Diffusion Transformer (MMDiT) architecture, where one branch synthesizes video and the other generates temporally aligned audio, while sharing a powerful text encoder based on the Multimodal Large Language Models (MLLM). SkyReels V4 accepts rich multi modal instructions, including text, images, video clips, masks, and audio references. By combining the MLLMs multi modal instruction following capability with in context learning in the video branch MMDiT, the model can inject fine grained visual guidance under complex conditioning, while the audio branch MMDiT simultaneously leverages audio references to guide sound generation. On the video side, we adopt a channel concatenation formulation that unifies a wide range of inpainting style tasks, such as image to video, video extension, and video editing under a single interface, and naturally extends to vision referenced inpainting and editing via multi modal prompts. SkyReels V4 supports up to 1080p resolution, 32 FPS, and 15 second duration, enabling high fidelity, multi shot, cinema level video generation with synchronized audio. To make such high resolution, long-duration generation computationally feasible, we introduce an efficiency strategy: Joint generation of low resolution full sequences and high-resolution keyframes, followed by dedicated super-resolution and frame interpolation models. To our knowledge, SkyReels V4 is the first video foundation model that simultaneously supports multi-modal input, joint video audio generation, and a unified treatment of generation, inpainting, and editing, while maintaining strong efficiency and quality at cinematic resolutions and durations.