Paper 2601.06943
Watching, Reasoning, and Searching: A Video Deep Research Benchmark on Open Web for Agentic Video Reasoning
- Published
- Jan 2026
- Research lab
- Independent
- Citations
- 6
- GitHub
- 160 stars
01 In brief
Summary
The paper introduces VideoDR, the first benchmark for video deep research, which requires models to extract multi-frame visual anchors from videos, perform interactive web searches, and conduct multi-hop reasoning over combined video-web evidence to answer open-domain factoid questions.
The benchmark comprises 500 samples across six semantic domains, with rigorous quality control ensuring that questions cannot be answered by video or web alone.
Human testing shows a mean success rate of 40.22%, with difficulty stratified by human performance.
Evaluations of six multimodal models under Workflow and Agentic paradigms reveal that Agentic is not consistently superior; its effectiveness depends on the model's ability to maintain initial video anchors over long retrieval chains.
Gemini-3.1-pro-preview leads with 53.60% accuracy under Agentic, while GPT-5.4 achieves 47.40%.
Error analysis identifies categorical and numerical errors as major failure modes, and goal drift and long-horizon consistency are highlighted as core bottlenecks.
The study concludes that Workflow provides stable textual anchors, reducing drift risk, whereas Agentic's direct video access demands stronger long-term state consistency.
The dataset and code are publicly available.
02 From the paper
Abstract
In real-world video question answering scenarios, videos often provide only localized visual cues, while verifiable answers are distributed across the open web; models therefore need to jointly perform cross-frame clue extraction, iterative retrieval, and multi-hop reasoning-based verification. To bridge this gap, we construct the first video deep research benchmark, VideoDR. VideoDR centers on video-conditioned open-domain video question answering, requiring cross-frame visual anchor extraction, interactive web retrieval, and multi-hop reasoning over joint video-web evidence; through rigorous human annotation and quality control, we obtain high-quality video deep research samples spanning six semantic domains. We evaluate multiple closed-source and open-source multimodal large language models under both the Workflow and Agentic paradigms, and the results show that Agentic is not consistently superior to Workflow: its gains depend on a model's ability to maintain the initial video anchors over long retrieval chains. Further analysis indicates that goal drift and long-horizon consistency are the core bottlenecks. In sum, VideoDR provides a systematic benchmark for studying video agents in open-web settings and reveals the key challenges for next-generation video deep research agents.