Paper 2606.05922
Evolving Agents in the Dark: Retrospective Harness Optimization via Self-Preference
- Published
- Jun 2026
- Research lab
- Independent
- Citations
- 1
- GitHub
- 48 stars
01 In brief
Summary
The paper introduces Retrospective Harness Optimization (RHO), a self-supervised method to improve an AI agent's harness (skills, tools, workflows) using only past trajectories, without ground-truth labels.
RHO selects a diverse, challenging coreset of past tasks via a determinantal point process, re-solves them in parallel, and extracts improvement signals from self-validation (within-trajectory correctness) and self-consistency (cross-trajectory agreement).
It then generates candidate harness updates and selects the best via pairwise self-preference.
Evaluated on SWE-Bench Pro, Terminal-Bench 2, and GAIA-2, a single RHO round improved SWE-Bench Pro pass rate from 59% to 78% without external grading.
RHO outperformed feedback-free baselines (Dynamic Cheatsheet, ReasoningBank, Sleep-time Compute) and matched validation-feedback Meta-Harness at higher compute.
The optimized harness added tools and skills targeting past failures, shifting agent behavior toward verification and execution, sustaining accuracy in long-horizon tasks.
Ablations showed both diagnostic signals and the DPP coreset (balancing difficulty and diversity) are crucial.
RHO operates without labels, edits the full harness, and runs in a single pass, unlike prior methods.
02 From the paper
Abstract
AI agents rely on a harness of skills, tools, and workflows to solve complex problems. Continually improving this harness is essential for adapting to new tasks. However, existing optimization methods typically require ground-truth validation sets, yet such labeled data is difficult to acquire in practical deployment settings. To address this problem, we introduce Retrospective Harness Optimization (RHO), a self-supervised method that optimizes the agent harness using only past trajectories. Specifically, RHO selects a diverse coreset of challenging tasks from past trajectories and re-solves them in parallel. The agent analyzes these rollouts using self-validation and self-consistency, then generates candidate harness updates and selects the most effective one by its own pairwise self-preference. We evaluate RHO across three diverse domains, spanning software engineering, technical work, and knowledge work. Notably, a single optimization round improves the pass rate on SWE-Bench Pro from 59% to 78% without any external grading. Furthermore, our analysis demonstrates that RHO effectively targets prior failure modes. As a result, the optimized harness alters the agent's behavior patterns and sustains higher accuracy during long-horizon sessions.