The Year in AI Papers
The year/Independent research

Paper 2601.17737

The Script is All You Need: An Agentic Framework for Long-Horizon Dialogue-to-Cinematic Video Generation

Chenyu Mu, Xin He, Qu Yang, Wanshun Chen, Jiadi Yao, Huang Liu, Zihao Yi, Bo Zhao, Xingyu Chen, Ruotian Ma, Fanghua Ye, Erkun Yang, et al.

Published
Jan 2026
Research lab
Independent
Citations
3
GitHub
Not linked

01 In brief

Summary

This paper introduces an agentic framework for generating long-horizon cinematic videos from dialogue.

The framework comprises three agents: ScripterAgent, which translates dialogue into detailed cinematic scripts; DirectorAgent, which orchestrates video generation using a cross-scene continuous generation strategy with frame-anchoring to ensure temporal coherence; and CriticAgent, which evaluates both script and video quality.

The authors construct ScriptBench, a benchmark of 1,750 instances with multimodal context, and train ScripterAgent using a two-stage paradigm (SFT followed by GRPO with a hybrid reward).

Experiments show that the full ScripterAgent outperforms baselines in script quality (e.g., Dramatic Tension 4.1 vs.

3.7) and that using its scripts improves all tested video models (e.g., Script Faithfulness up to +0.4).

A new metric, Visual-Script Alignment (VSA), confirms enhanced temporal-semantic coherence (over 7 points).

The study also reveals a trade-off between visual spectacle and script adherence in state-of-the-art models.

02 From the paper

Abstract

Recent advances in video generation have produced models capable of synthesizing stunning visual content from simple text prompts. However, these models struggle to generate long-form, coherent narratives from high-level concepts like dialogue, revealing a ``semantic gap'' between a creative idea and its cinematic execution. To bridge this gap, we introduce a novel, end-to-end agentic framework for dialogue-to-cinematic-video generation. Central to our framework is ScripterAgent, a model trained to translate coarse dialogue into a fine-grained, executable cinematic script. To enable this, we construct ScriptBench, a new large-scale benchmark with rich multimodal context, annotated via an expert-guided pipeline. The generated script then guides DirectorAgent, which orchestrates state-of-the-art video models using a cross-scene continuous generation strategy to ensure long-horizon coherence. Our comprehensive evaluation, featuring an AI-powered CriticAgent and a new Visual-Script Alignment (VSA) metric, shows our framework significantly improves script faithfulness and temporal fidelity across all tested video models. Furthermore, our analysis uncovers a crucial trade-off in current SOTA models between visual spectacle and strict script adherence, providing valuable insights for the future of automated filmmaking.