Paper 2607.07675
Scaling Mixture-of-Experts Video Pretraining for Embodied Intelligence
- Published
- Jul 2026
- Research lab
- Independent
- Citations
- 3
- GitHub
- 908 stars
01 In brief
Summary
LingBot-Video is a DiT-based video pretraining paradigm for embodied intelligence, introduced as the first large-scale open-source Mixture-of-Experts (MoE) video foundation model.
It addresses domain mismatch in video generation by using a sparse MoE framework for better capacity-efficiency trade-off, a data profiling engine that augments internet videos with robot-oriented footage (manipulation, navigation, egocentric), and a multi-dimensional reward system enforcing physical rationality and task completion.
The architecture uses a single-stream diffusion transformer with multi-modal 3D RoPE, QK-Norm, and adaLN-single modulation.
Scaling experiments show MoE 13B-A1.4B outperforms Dense 1.3B, with predictable scaling up to 120B parameters and inference speed ratios up to 3.18x faster than Dense 30B at 1M tokens.
Training uses a five-stage progressive curriculum from 192p images to 1080p refinement.
Post-training includes GRPO with six reward models (vision quality, text-video alignment, dynamic degree, motion coherence, human-motion consistency, physical plausibility) and negative-aware finetuning with real videos.
Evaluations show state-of-the-art performance on RBench (0.620 avg) and Physics-IQ Verified (40.4), with user study advantages in TI2V.
The model is also post-trained as an action-conditioned world model (LingBot-Video-A2V) and distilled for few-step inference.
02 From the paper
Abstract
Despite the recent promise in robot control, video generative models suffer from a domain mismatch due to their primary focus on content creation. For example, their design inherently prioritizes visual fidelity and creativity over computational efficiency and physical realism. In this work, we present LingBot-Video, a DiT-based video pretraining paradigm specifically tailored for embodied intelligence. From the architecture perspective, we adopt the Mixture-of-Experts (MoE), instead of dense, framework to achieve a better trade-off between modeling capacity and inference efficiency, and manage to scale it up from scratch. From the data perspective, we construct a data profiling engine that augments standard internet videos with extensive robot-oriented footage, encompassing manipulation, navigation, and egocentric perspectives, to equip the base model with an intrinsic understanding of actions and world dynamics. From the training perspective, we develop a multi-dimensional reward system to enforce the alignment regarding physical rationality and task completion, going beyond standard criteria such as aesthetics, prompt-following, and motion consistency. Comprehensive evaluations validate its performance and efficiency as a video foundation model. We contribute LingBot-Video as the inaugural large-scale, open-source MoE video foundation model to the community, in a pioneering effort to bridge digital creativity and physical actuation.