Paper 2510.05096
Paper2Video: Automatic Video Generation from Scientific Papers
- Published
- Oct 2025
- Research lab
- Independent
- Citations
- 24
- GitHub
- 2.3K stars
01 In brief
Summary
The paper introduces Paper2Video, the first benchmark of 101 research papers paired with author-created presentation videos, slides, and speaker metadata, along with four evaluation metrics: Meta Similarity, PresentArena, PresentQuiz, and IP Memory.
It also proposes PaperTalker, a multi-agent framework that generates presentation videos from papers, integrating slide generation with a novel Tree Search Visual Choice for layout refinement, cursor grounding, subtitling, speech synthesis, and talking-head rendering, with slide-wise parallel generation for efficiency.
Experiments show PaperTalker outperforms baselines, achieving 10% higher PresentQuiz accuracy than human-made videos and comparable user ratings, while reducing production time by 6x.
The dataset, agent, and code are open-sourced at https://github.com/showlab/Paper2Video.
The work addresses challenges of long-context inputs, multi-modal information, and multi-channel coordination in academic video generation, establishing a practical step toward automated scholarly communication.
The benchmark includes papers from ML, CV, and NLP conferences, with presentations averaging 16 slides and 6 minutes 15 seconds.
The metrics evaluate information conveyance, audience engagement, and author visibility, with PaperTalker achieving the highest scores across all metrics.
The framework's components are ablated, showing the cursor improves grounding accuracy and the tree search improves slide design quality.
Human evaluation ranks PaperTalker second only to human-made videos, outperforming all other baselines.
The work aims to advance AI for Research and support scalable scholarly communication.
02 From the paper
Abstract
Academic presentation videos have become an essential medium for research communication, yet producing them remains highly labor-intensive, often requiring hours of slide design, recording, and editing for a short 2 to 10 minutes video. Unlike natural video, presentation video generation involves distinctive challenges: inputs from research papers, dense multi-modal information (text, figures, tables), and the need to coordinate multiple aligned channels such as slides, subtitles, speech, and human talker. To address these challenges, we introduce Paper2Video, the first benchmark of 101 research papers paired with author-created presentation videos, slides, and speaker metadata. We further design four tailored evaluation metrics--Meta Similarity, PresentArena, PresentQuiz, and IP Memory--to measure how videos convey the paper's information to the audience. Building on this foundation, we propose PaperTalker, the first multi-agent framework for academic presentation video generation. It integrates slide generation with effective layout refinement by a novel effective tree search visual choice, cursor grounding, subtitling, speech synthesis, and talking-head rendering, while parallelizing slide-wise generation for efficiency. Experiments on Paper2Video demonstrate that the presentation videos produced by our approach are more faithful and informative than existing baselines, establishing a practical step toward automated and ready-to-use academic video generation. Our dataset, agent, and code are available at https://github.com/showlab/Paper2Video.