Paper 2607.13285
Harness Handbook: Making Evolving Agent Harnesses Readable,Navigable, and Editable
- Published
- Jul 2026
- Research lab
- Independent
- Citations
- 1
- GitHub
- 292 stars
01 In brief
Summary
The paper introduces Harness Handbook, a behavior-centric representation for agent harnesses that links system behaviors to their distributed source implementations, addressing the challenge of behavior localization during harness evolution.
It is constructed automatically via static analysis and LLM-assisted structuring, organizing knowledge into a three-level hierarchy (L1 system overview, L2 component overview, L3 unit deep dive) plus a state-register view.
The modification workflow uses Behavior-Guided Progressive Disclosure (BGPD) to navigate from high-level behavior descriptions to verified source locations, followed by edit planning, execution, and automatic resynchronization after code changes.
Evaluated on two open-source harnesses (Terminus-2 and Codex) with 30 modification requests each, Handbook-Assisted planning improved plan quality (win rates +10.0 and +18.9 percentage points) while reducing planner token use (12.7% and 8.6% lower).
It also enabled a weaker planner to match stronger models' localization accuracy, improving all 24 file- and symbol-level Recall, Precision, and F1 comparisons against reference plans.
Gains persisted across request types and difficulty levels, with the largest improvements for scattered, rarely executed, and cross-module changes.
The work highlights that evolving complex agentic systems depends on determining where edits should be made, not just generating them.
02 From the paper
Abstract
The capability of a modern AI agent depends not only on its foundation model but also on its harness, which constructs prompts, manages state, invokes tools, and coordinates execution. As models, APIs, environments, and requirements evolve, the harness must be continually modified. Before such a change can be made, a developer or coding agent must identify all code locations that implement the target behavior. This is difficult because production harnesses are large, tightly coupled, and behaviorally distributed, while modification requests describe what the system should do and repositories are organized by files and modules. Code search, repository indexing, and long-context processing ease inspection, but still leave this behavior-to-code mapping to be recovered by hand. Behavior localization is therefore a central bottleneck in harness evolution. We introduce the Harness Handbook, a behavior-centric representation synthesized automatically from a harness codebase via static analysis and LLM-assisted structuring, linking each behavior to its corresponding source. We also introduce Behavior-Guided Progressive Disclosure (BGPD), which guides agents from high-level behaviors to relevant implementation details and verifies candidate locations against the current source. On diverse modification requests from two open-source harnesses, Handbook-Assisted planning improves behavior localization and edit-plan quality while using fewer planner tokens, with the largest gains on scattered sites, rarely executed paths, and cross-module interactions. Evolving complex agentic systems thus depends not only on generating edits, but also on determining where those edits should be made.