Paper 2602.10809
DeepImageSearch: Benchmarking Multimodal Agents for Context-Aware Image Retrieval in Visual Histories
- Published
- Feb 2026
- Research lab
- Independent
- Citations
- 3
- GitHub
- 87 stars
01 In brief
Summary
DeepImageSearch introduces a new paradigm for image retrieval, reformulating it as an agentic exploration task over visual histories.
The authors construct DISBench, a benchmark with 122 queries across 57 users and 109,467 photos, built via a human-model collaborative pipeline that mines latent spatiotemporal associations.
Queries are categorized into Intra-Event (locate an event then filter targets) and Inter-Event (verify recurring elements across events).
They propose ImageSeeker, a baseline agent framework with tools for retrieval, metadata filtering, and visual verification, plus a dual-memory system.
Experiments show state-of-the-art models achieve only 28.7 EM and 55.0 F1, while embedding-based retrieval performs near chance.
Ablations confirm all components contribute, with metadata tools and explicit memory most critical.
Error analysis identifies reasoning breakdowns as the dominant failure.
Test-time scaling improves performance but reveals challenges in reasoning path selection.
The work highlights the need for corpus-level contextual reasoning in next-generation retrieval systems.
02 From the paper
Abstract
Existing multimodal retrieval systems excel at semantic matching but implicitly assume that query-image relevance can be measured in isolation. This paradigm overlooks the rich dependencies inherent in realistic visual streams, where information is distributed across temporal sequences rather than confined to single snapshots. To bridge this gap, we introduce DeepImageSearch, a novel agentic paradigm that reformulates image retrieval as an autonomous exploration task. Models must plan and perform multi-step reasoning over raw visual histories to locate targets based on implicit contextual cues. We construct DISBench, a challenging benchmark built on interconnected visual data. To address the scalability challenge of creating context-dependent queries, we propose a human-model collaborative pipeline that employs vision-language models to mine latent spatiotemporal associations, effectively offloading intensive context discovery before human verification. Furthermore, we build a robust baseline using a modular agent framework equipped with fine-grained tools and a dual-memory system for long-horizon navigation. Extensive experiments demonstrate that DISBench poses significant challenges to state-of-the-art models, highlighting the necessity of incorporating agentic reasoning into next-generation retrieval systems.