Paper 2509.24107
Fathom-DeepResearch: Unlocking Long Horizon Information Retrieval and Synthesis for SLMs
- Published
- Sep 2025
- Research lab
- Independent
- Citations
- 4
- GitHub
- 63 stars
01 In brief
Summary
Fathom-DeepResearch is an agentic system for long-horizon information retrieval and synthesis, composed of two 4B-parameter models built on Qwen3-4B.
Fathom-Search-4B handles evidence-based investigation via live web search and targeted page querying, trained with DUETQA (a ~5K-sample dataset generated via multi-agent self-play ensuring strict web-search dependence), RAPO (a zero-overhead GRPO extension with curriculum pruning, advantage scaling, and replay buffers to stabilize multi-turn RL), and a steerable step-level reward that classifies tool calls to control search breadth, depth, and horizon, enabling reliable tool use beyond 20 calls.
Fathom-Synthesizer-4B converts multi-turn search traces into citation-dense DeepResearch Reports using a plan-then-write protocol, trained on DEEPRESEARCH-SFT (2,500 synthetic questions distilled from GPT-5).
Evaluated on DeepSearch benchmarks (SimpleQA, FRAMES, WebWalker, Seal0, MuSiQue) and DeepResearch-Bench, the system achieves state-of-the-art performance in the open-weights category, outperforming many closed-source systems, and shows strong generalization to reasoning tasks (HLE, AIME-25, GPQA-Diamond, MedQA).
Ablations show RAPO outperforms GRPO, and the steerable reward improves accuracy and efficiency over vanilla rewards, though RAPO shows limited test-time scaling.
02 From the paper
Abstract
Tool-integrated reasoning has emerged as a key focus for enabling agentic applications. Among these, DeepResearch Agents have gained significant attention for their strong performance on complex, open-ended information-seeking tasks. We introduce Fathom-DeepResearch, an agentic system composed of two specialized models. The first is Fathom-Search-4B, a DeepSearch model trained from Qwen3-4B and optimized for evidence-based investigation through live web search and targeted webpage querying. Its training combines three advances: (i) DUETQA, a 5K-sample dataset generated via multi-agent self-play that enforces strict web-search dependence and heterogeneous source grounding; (ii) RAPO, a zero-overhead extension of GRPO that stabilizes multi-turn Reinforcement Learning with Verifiable Rewards through curriculum pruning, reward-aware advantage scaling, and per-prompt replay buffers; and (iii) a steerable step-level reward that classifies each tool call by cognitive behavior and marginal utility, enabling explicit control over search trajectory breadth, depth, and horizon. These improvements enable reliable extension of tool-calling beyond 20 calls when warranted. The second is Fathom-Synthesizer-4B, trained from Qwen3-4B, which converts multi-turn DeepSearch traces into structured, citation-dense DeepResearch Reports for comprehensive synthesis. Evaluated on DeepSearch benchmarks (SimpleQA, FRAMES, WebWalker, Seal0, MuSiQue) and DeepResearch-Bench, the system achieves state-of-the-art performance in the open-weights category while demonstrating strong generalization to diverse reasoning tasks including HLE, AIME-25, GPQA-Diamond, and MedQA.