Paper 2510.21618
DeepAgent: A General Reasoning Agent with Scalable Toolsets
- Published
- Oct 2025
- Research lab
- Independent
- Citations
- 63
- GitHub
- 1.1K stars
01 In brief
Summary
DeepAgent is an end-to-end deep reasoning agent that unifies autonomous thinking, tool discovery, and action execution within a single reasoning process, addressing limitations of predefined workflow-based agents.
It introduces an autonomous memory folding mechanism that compresses interaction history into structured episodic, working, and tool memories, enabling the agent to 'take a breath' and reconsider strategies.
For training, it proposes ToolPO, an end-to-end reinforcement learning method using LLM-simulated APIs for stability and tool-call advantage attribution for fine-grained credit assignment.
Experiments on eight benchmarks (ToolBench, API-Bank, TMDB, Spotify, ToolHop, ALFWorld, WebShop, GAIA, HLE) show DeepAgent consistently outperforms baselines, especially in open-set tool retrieval scenarios.
Key results include 89.0% on TMDB and 75.4% on Spotify (labeled tools), and 64.0% on ToolBench and 40.6% on ToolHop (open-set), with 32B models.
Ablations confirm the importance of ToolPO training, memory folding, tool simulation, and advantage attribution.
DeepAgent also scales effectively with larger backbones and action limits.
02 From the paper
Abstract
Large reasoning models have demonstrated strong problem-solving abilities, yet real-world tasks often require external tools and long-horizon interactions. Existing agent frameworks typically follow predefined workflows, which limit autonomous and global task completion. In this paper, we introduce DeepAgent, an end-to-end deep reasoning agent that performs autonomous thinking, tool discovery, and action execution within a single, coherent reasoning process. To manage long-horizon interactions, we introduce an autonomous memory folding mechanism that compresses past interactions into structured episodic, working, and tool memories, reducing error accumulation while preserving critical information. To teach general-purpose tool use efficiently and stably, we develop an end-to-end reinforcement learning strategy, namely ToolPO, that leverages LLM-simulated APIs and applies tool-call advantage attribution to assign fine-grained credit to the tool invocation tokens. Extensive experiments on eight benchmarks, including general tool-use tasks (ToolBench, API-Bank, TMDB, Spotify, ToolHop) and downstream applications (ALFWorld, WebShop, GAIA, HLE), demonstrate that DeepAgent consistently outperforms baselines across both labeled-tool and open-set tool retrieval scenarios. The code and demo are available at https://github.com/RUC-NLPIR/DeepAgent.