Paper 2509.17627
OmniInsert: Mask-Free Video Insertion of Any Reference via Diffusion Transformer Models
- Published
- Sep 2025
- Research lab
- Independent
- Citations
- 9
- GitHub
- 162 stars
01 In brief
Summary
OmniInsert is a unified framework for mask-free video insertion, allowing users to insert single or multiple reference subjects into a source video based on a text prompt.
It addresses three key challenges: data scarcity, subject-scene equilibrium, and insertion harmonization.
To tackle data scarcity, the authors propose InsertPipe, a data pipeline with three sub-pipes (RealCapture, SynthGen, SimInteract) that automatically generates diverse training data.
For subject-scene equilibrium, they introduce a Condition-Specific Feature Injection (CFI) mechanism, a four-stage Progressive Training (PT) strategy, and a Subject-Focused Loss (SL).
To improve insertion harmonization, they propose Insertive Preference Optimization (IPO) and a Context-Aware Rephraser (CAR) module.
They also introduce InsertBench, a benchmark of 120 videos with selected subjects and prompts.
Experiments show OmniInsert outperforms commercial solutions like Pika-Pro and Kling on metrics such as CLIP-I, DINO-I, and ViCLIP-T, and in user studies.
The model uses a Diffusion Transformer with LoRA, trained in four phases, and generates a 5-second 480P video in about 90 seconds.
02 From the paper
Abstract
Recent advances in video insertion based on diffusion models are impressive. However, existing methods rely on complex control signals but struggle with subject consistency, limiting their practical applicability. In this paper, we focus on the task of Mask-free Video Insertion and aim to resolve three key challenges: data scarcity, subject-scene equilibrium, and insertion harmonization. To address the data scarcity, we propose a new data pipeline InsertPipe, constructing diverse cross-pair data automatically. Building upon our data pipeline, we develop OmniInsert, a novel unified framework for mask-free video insertion from both single and multiple subject references. Specifically, to maintain subject-scene equilibrium, we introduce a simple yet effective Condition-Specific Feature Injection mechanism to distinctly inject multi-source conditions and propose a novel Progressive Training strategy that enables the model to balance feature injection from subjects and source video. Meanwhile, we design the Subject-Focused Loss to improve the detailed appearance of the subjects. To further enhance insertion harmonization, we propose an Insertive Preference Optimization methodology to optimize the model by simulating human preferences, and incorporate a Context-Aware Rephraser module during reference to seamlessly integrate the subject into the original scenes. To address the lack of a benchmark for the field, we introduce InsertBench, a comprehensive benchmark comprising diverse scenes with meticulously selected subjects. Evaluation on InsertBench indicates OmniInsert outperforms state-of-the-art closed-source commercial solutions. The code will be released.