Paper 2604.04949
Learning to Retrieve from Agent Trajectories
- Published
- Mar 2026
- Research lab
- Independent
- Citations
- 2
- GitHub
- 56 stars
01 In brief
Summary
This paper introduces LRAT (Learning to Retrieve from Agent Trajectories), a framework for training retrieval models directly from the interaction data of LLM-powered search agents.
The authors argue that traditional human-centric retrieval training is misaligned with agentic search, where agents issue intermediate queries and consume results in multi-turn reasoning loops.
Analysis of deep research agent trajectories reveals that browsing actions are a necessary condition for task success, unbrowsed documents serve as reliable negatives without position bias, and post-browse reasoning length indicates relevance intensity.
LRAT mines supervision from search-browse transitions, filters positives using an LLM judge on post-browse reasoning, and incorporates relevance intensity via reasoning-length-aware weighting in a contrastive loss.
Experiments on InfoSeek-Eval and BrowseComp-Plus show consistent improvements in success rate, evidence recall, and execution efficiency across six agent backbones (4B to 358B parameters) and two retriever backbones.
Ablations confirm each component's contribution, and simulations show LRAT supports a self-improving data flywheel, with gains even from incorrect trajectories.
The work positions agent trajectories as a scalable supervision source for retrieval in the agentic search era.
02 From the paper
Abstract
Information retrieval (IR) systems have traditionally been designed and trained for human users, with learning-to-rank methods relying heavily on large-scale human interaction logs such as clicks and dwell time. With the rapid emergence of large language model (LLM) powered search agents, however, retrieval is increasingly consumed by agents rather than human beings, and is embedded as a core component within multi-turn reasoning and action loops. In this setting, retrieval models trained under human-centric assumptions exhibit a fundamental mismatch with the way agents issue queries and consume results. In this work, we argue that retrieval models for agentic search should be trained directly from agent interaction data. We introduce learning to retrieve from agent trajectories as a new training paradigm, where supervision is derived from multi-step agent interactions. Through a systematic analysis of search agent trajectories, we identify key behavioral signals that reveal document utility, including browsing actions, unbrowsed rejections, and post-browse reasoning traces. Guided by these insights, we propose LRAT, a simple yet effective framework that mines high-quality retrieval supervision from agent trajectories and incorporates relevance intensity through weighted optimization. Extensive experiments on both in-domain and out-of-domain deep research benchmarks demonstrate that retrievers trained with LRAT consistently improve evidence recall, end-to-end task success, and execution efficiency across diverse agent architectures and scales. Our results highlight agent trajectories as a practical and scalable supervision source, pointing to a promising direction for retrieval in the era of agentic search.