How to audit which harness edits were adopted and why with harness2

Authorization notice. harness2 is dual-use. Only use it on software, sites and systems you own or have explicit permission to test.

Who it is forAgent builder who wants to copy the winning edits into a hand-maintained harness instead of re-running refinement

The problemYou get a better score but do not know which of the eight editable components changed or what lesson justified it.

The outcomeA short list of concrete edits (system prompt, guardrails, memory, tool guidance and so on) with their stated evidence, ready to port by hand.

What you need first

• Linux, Python 3.12 and uv for installation
• No credentials needed for the local demo (harness2-demo)
• Model-backed runs need Node.js 20+, the Google Cloud CLI, a Vertex AI project and model IDs in .env, and gcloud application-default login
• LAB needs Pandoc and Poppler; WorkBuddy-Bench needs Docker
• Model-backed benchmark runs incur API costs

Steps

1. Finish a refinement run, for example: harness2 --bench lab --k 1 --mode parallel --only-domains antitrust-competition --max-tasks 1
2. Open the benchmark folder under runs/ and find the adopted harnesses and reflection notes.
3. Compare the adopted harness against the base harness, component by component.
4. Note which edits came from the general candidate versus the domain-specific candidate.
5. Copy the edits you trust into your own agent setup.

Prompt for your agent

After a google-research/harness2 run on <benchmark>, open the runs/ folder and list every adopted harness edit grouped by component (system prompt, guardrails, memory, tool guidance, subagents, skills, scripts, plugins). Verify each edit has a matching reflection note before recommending I port it.

Caveat. The README does not document the exact file layout inside runs/, so inspect the actual output folders rather than assuming a structure, and treat a single run's lessons as hypotheses.

• Markdown version
• JSON version