Paper 2607.21655
Progress Reward Modeling for Robotic Learning: A Comprehensive Survey
- Published
- Jul 2026
- Research lab
- Independent
- Citations
- 0
- GitHub
- 69 stars
01 In brief
Summary
This survey provides a unified framework for progress reward modeling in robotic learning, addressing the fragmentation in the field.
It organizes the domain into three connected steps: the interface of progress models (input/output structure), the methods for constructing progress rewards, and the data and benchmarks used for evaluation.
The interface section covers how current task state is represented (single observation, temporal context, relational comparison, state-access), how task goals are specified (language, vision, structured/programmatic), and the output forms (state-wise scores, progress deltas, rankings, executable reward functions).
The methods section identifies four paradigms: frozen foundation-model scoring, learning from temporal/relative supervision, instruction-tuned progress prediction, and programmatic reward construction.
The data and benchmarks section examines progress supervision construction (human-driven, human-in-the-loop, fully automated) and evaluation goals (progress fidelity, robustness/generalization, downstream utility).
The survey concludes with limitations including coarse-grained estimation, fixed-rate progress assumptions, inference latency, and lack of long-horizon memory, and suggests future directions such as finer resolution, adaptive temporal modeling, efficient online inference, and explicit task memory.
The authors aim to connect what a progress model is, how it is built, and how its quality is validated, providing a comprehensive reference for researchers and practitioners in robotic learning.
02 From the paper
Abstract
Robotic learning takes place in dynamic environments with large behavior spaces. A terminal success signal only tells the robot whether the task is completed. It does not explain whether the current behavior is making progress, remaining unchanged, or undoing earlier progress. For this reason, recent studies have increasingly explored progress rewards that provide feedback during task execution. However, the current literature lacks a shared framework. Existing methods use different observations, goal specifications, output signals, supervision sources, and evaluation protocols. This makes it difficult to compare them and understand what their results actually validate. In this survey, we provide a unified view of progress reward modeling for robotic learning. We organize the field in three connected steps. We first study the interface of a progress model. This defines the problem from the outside by asking what information the model receives and what form of progress signal it produces. We then move inside the model and study the methods used to construct this signal. This reveals the different assumptions and mechanisms behind progress estimation and reward generation. Finally, we examine the data and benchmarks that support these methods. This shows how progress supervision is obtained and what different evaluations actually measure. Together, these three perspectives connect what a progress model is, how it is built, and how its quality is validated. We further summarize the main limitations of current approaches and discuss future research directions.