Paper 2606.02060
Where Do Deep-Research Agents Go Wrong? Span-Level Error Localization in Agent Trajectories
- Published
- Jun 2026
- Research lab
- Independent
- Citations
- 0
- GitHub
- 22 stars
01 In brief
Summary
This paper introduces TELBENCH, a benchmark for span-level error localization in deep-research agent trajectories, and DRIFT, a claim-centric auditing framework.
The authors collected 2,790 real trajectories from two agent frameworks (MiroFlow, OAgent), three backbone models (GPT-5, Gemini-2.5-Pro, Claude-Sonnet-4.5), and three benchmarks (GAIA, XBench, BrowseComp), converting them into semantic spans and annotating harmful errors via LLM-assisted expert review.
TELBENCH contains 1,000 verified instances (600 easy, 400 hard) with an average of 11.95 spans per trajectory.
DRIFT audits claims by tracking their introduction, support status (direct, weak, missing, conflicting), and dependency propagation, outperforming bare LLM prompting and generic agentic auditors (Codex, Claude Code) by up to 30 percentage points in F1 and first-error accuracy across five model families.
Key findings include that process errors are not equivalent to final-answer failure (36.9% of successful trajectories contain errors), error mechanisms are stage-dependent, and scaling alone is insufficient for reliable diagnosis.
First-error localization remains challenging, especially on hard instances.
02 From the paper
Abstract
Deep-research agents solve tasks through long trajectories of search, tool use, evidence inspection, and answer synthesis. Evaluation based on final answers shows whether an agent succeeds, but not which parts of the trajectory make the answer unreliable. We study span-level error localization for deep-research agents. We collect 2,790 real trajectories from two agent frameworks, three backbone models, and three benchmarks, convert raw logs into semantic spans, and annotate harmful error spans through LLM-assisted expert review. From these annotations, we build TELBench, a 1,000-instance benchmark for identifying error spans among normal exploration, failed searches, tentative hypotheses, and harmless noise. We further propose DRIFT, a claim-centric auditing framework that tracks agent claims, checks their support in trajectory evidence, and marks spans where unsupported or conflicting claims affect the answer path. Experiments across model families and auditing frameworks show that DRIFT improves span-level error localization and first-error accuracy by up to 30 percentage points. Our work provides a process-level view of reliability in deep-research agents.