Paper 2607.27380
VideoCoCo: Code-as-CoT for Physically-Consistent Video Generation via an Agentic Dual-Engine System
- Published
- Jul 2026
- Research lab
- Independent
- Citations
- 0
- GitHub
- Not linked
01 In brief
Summary
VideoCoCo is an agentic dual-engine framework for physically consistent text-to-video generation.
It uses executable Blender code as a process-level chain of thought.
A coding agent synthesizes a Blender program from a text prompt, which is executed in a sandbox to produce a deterministic, low-fidelity spatiotemporal draft.
A generative video engine then transforms this draft into a photorealistic video via draft-conditioned editing, guided by an instruction agent that composes appearance-focused editing instructions.
To train the editor, the authors construct VideoCoCo-3K, a dataset of 3,000 draft–instruction–target triplets generated using a teacher-based pipeline with Seedance 2.0.
Experiments show that VideoCoCo improves the OmniWeaving baseline from 0.475 to 0.558 on PhyGenBench and from 52.18% to 77.88% on VBench-2.0, achieving the best average scores on both benchmarks.
Ablations show that both the executable drafting and editor adaptation contribute, with LoRA tuning outperforming full fine-tuning.
The approach decouples process-level reasoning from visual realization, making the intermediate representation inspectable and controllable.
Limitations include added inference latency and reliance on the Blender simulator's expressiveness.
02 From the paper
Abstract
Text-to-video models have achieved remarkable visual quality, yet they still struggle to generate physically consistent dynamics because the temporal evolution of a scene must be inferred implicitly from a highly compressed text prompt. Existing chain-of-thought approaches introduce intermediate plans or visual states, but these representations are typically non-executable or temporally sparse, limiting their ability to instantiate and control the complete spatiotemporal process. To address this limitation, we introduce VideoCoCo, an agentic dual-engine framework in which executable Blender code serves as a process-level chain of thought. Given a text prompt, a coding agent synthesizes a Blender program that explicitly specifies the scene and its temporal evolution. The executable simulation engine runs the program to produce a deterministic spatiotemporal draft, which is subsequently transformed into a photorealistic video by a generative video engine through draft-conditioned editing. This decomposition separates process-level reasoning from high-fidelity visual realization. To adapt the video editor to simulated drafts, we construct VideoCoCo-3K, a curated dataset of draft-instruction-target triplets. VideoCoCo improves the OmniWeaving baseline from 0.475 to 0.558 on PhyGenBench and from 52.18 to 77.88 on VBench-2.0, achieving the best average score on both benchmarks. These results demonstrate that executable code provides an effective, controllable, and inspectable intermediate representation for physically consistent video generation.