Paper 2509.02547
The Landscape of Agentic Reinforcement Learning for LLMs: A Survey
- Published
- Sep 2025
- Research lab
- Independent
- Citations
- 182
- GitHub
- 1.9K stars
01 In brief
Summary
This survey formalizes Agentic Reinforcement Learning (Agentic RL), a paradigm where LLMs are treated as learnable policies in sequential decision-making loops, contrasting it with conventional single-step LLM RL.
It proposes a twofold taxonomy: one based on core agentic capabilities (planning, tool use, memory, reasoning, self-improvement, perception) and another on applications (search, code, math, GUI, vision, embodied, multi-agent).
The survey argues that RL is the critical mechanism for transforming these capabilities from static modules into adaptive behaviors.
It consolidates open-source environments, benchmarks, and frameworks, and discusses open challenges including trustworthiness, scaling training and environments, and the mechanistic debate on RL's effects.
The work synthesizes over 500 recent studies to chart the field's landscape and future directions for scalable, general-purpose AI agents.
The survey is structured to progress from conceptual foundations (MDP/POMDP formalization) to practical implementations, providing a comprehensive resource for researchers.
It also highlights the shift from preference-based RFT to task-specific, dynamic-environment agentic RL, emphasizing the need for long-horizon credit assignment and adaptive interaction.
The authors aim to bridge the gap between isolated studies by providing a unified treatment of Agentic RL, connecting theoretical foundations with algorithmic approaches and practical systems.
They also provide a practical compendium of open-source resources to support and accelerate future research in this rapidly evolving field.
The survey concludes by discussing open challenges and outlining promising future directions for scalable, general-purpose agentic intelligence.
The work is published in Transactions on Machine Learning Research (01/2026) and is available on OpenReview.
The survey's primary focus is on how…
02 From the paper
Abstract
The emergence of agentic reinforcement learning (Agentic RL) marks a paradigm shift from conventional reinforcement learning applied to large language models (LLM RL), reframing LLMs from passive sequence generators into autonomous, decision-making agents embedded in complex, dynamic worlds. This survey formalizes this conceptual shift by contrasting the degenerate single-step Markov Decision Processes (MDPs) of LLM-RL with the temporally extended, partially observable Markov decision processes (POMDPs) that define Agentic RL. Building on this foundation, we propose a comprehensive twofold taxonomy: one organized around core agentic capabilities, including planning, tool use, memory, reasoning, self-improvement, and perception, and the other around their applications across diverse task domains. Central to our thesis is that reinforcement learning serves as the critical mechanism for transforming these capabilities from static, heuristic modules into adaptive, robust agentic behavior. To support and accelerate future research, we consolidate the landscape of open-source environments, benchmarks, and frameworks into a practical compendium. By synthesizing over five hundred recent works, this survey charts the contours of this rapidly evolving field and highlights the opportunities and challenges that will shape the development of scalable, general-purpose AI agents.