Paper 2512.20618
LongVideoAgent: Multi-Agent Reasoning with Long Videos
- Published
- Dec 2025
- Research lab
- Independent
- Citations
- 21
- GitHub
- 126 stars
01 In brief
Summary
LongVideoAgent is a multi-agent framework for long-video question answering.
A master LLM coordinates a grounding agent to localize question-relevant segments and a vision agent to extract targeted visual observations.
The master agent plans with a step limit and is trained with reinforcement learning (GRPO) to encourage concise, correct, and efficient cooperation.
The system outperforms non-agent baselines on the proposed LongTVQA and LongTVQA+ datasets, which aggregate TVQA/TVQA+ clips into episode-level sequences.
Ablations show that grounding and vision are essential, increasing the step limit from 2 to 5 improves accuracy, larger evidence windows help, and stronger vision models yield higher accuracy.
Reinforcement learning further boosts performance, especially for smaller open-source models.
02 From the paper
Abstract
Recent advances in multimodal LLMs and systems that use tools for long-video QA point to the promise of reasoning over hour-long episodes. However, many methods still compress content into lossy summaries or rely on limited toolsets, weakening temporal grounding and missing fine-grained cues. We propose a multi-agent framework in which a master LLM coordinates a grounding agent to localize question-relevant segments and a vision agent to extract targeted textual observations. The master agent plans with a step limit, and is trained with reinforcement learning to encourage concise, correct, and efficient multi-agent cooperation. This design helps the master agent focus on relevant clips via grounding, complements subtitles with visual detail, and yields interpretable trajectories. On our proposed LongTVQA and LongTVQA+ which are episode-level datasets aggregated from TVQA/TVQA+, our multi-agent system significantly outperforms strong non-agent baselines. Experiments also show reinforcement learning further strengthens reasoning and planning for the trained agent. Code and data will be shared at https://longvideoagent.github.io/.