Paper 2512.16776
Kling-Omni Technical Report
- Published
- Dec 2025
- Research lab
- Independent
- Citations
- 41
- GitHub
- Not linked
01 In brief
Summary
Kling-Omni is a generalist generative framework from Kuaishou Technology that unifies video generation, editing, and reasoning into a single end-to-end system.
It introduces Multi-modal Visual Language (MVL) as an interaction paradigm, combining text, images, and videos into a unified representation.
The architecture includes a Prompt Enhancer (PE) based on an MLLM, an Omni-Generator, and a Multimodal Super-Resolution module.
Training uses a multi-stage strategy: pre-training, supervised fine-tuning, and reinforcement learning with DPO.
Inference is accelerated via distillation (reducing NFE from 150 to 10), hybrid parallelism, FP8 quantization, and caching.
The data system includes real-world and synthetic data with a three-tier filtering pipeline.
Evaluations on the OmniVideo-1.0 benchmark show Kling-Omni outperforms Veo 3.1 and Runway-Aleph in reference-based generation and video editing, with an overall GSB of 247%.
The model supports features like element library reference, temporal narrative, high-degree-of-freedom editing, and task composition.
It also demonstrates potential in visual prompt understanding and reasoning-enhanced generation, though these are not yet available online.
02 From the paper
Abstract
We present Kling-Omni, a generalist generative framework designed to synthesize high-fidelity videos directly from multimodal visual language inputs. Adopting an end-to-end perspective, Kling-Omni bridges the functional separation among diverse video generation, editing, and intelligent reasoning tasks, integrating them into a holistic system. Unlike disjointed pipeline approaches, Kling-Omni supports a diverse range of user inputs, including text instructions, reference images, and video contexts, processing them into a unified multimodal representation to deliver cinematic-quality and highly-intelligent video content creation. To support these capabilities, we constructed a comprehensive data system that serves as the foundation for multimodal video creation. The framework is further empowered by efficient large-scale pre-training strategies and infrastructure optimizations for inference. Comprehensive evaluations reveal that Kling-Omni demonstrates exceptional capabilities in in-context generation, reasoning-based editing, and multimodal instruction following. Moving beyond a content creation tool, we believe Kling-Omni is a pivotal advancement toward multimodal world simulators capable of perceiving, reasoning, generating and interacting with the dynamic and complex worlds.