How to resume a long refinement run after it stops midway with harness2

Authorization notice. harness2 is dual-use. Only use it on software, sites and systems you own or have explicit permission to test.

Who it is forResearcher running a larger JobBench or LAB sweep that was interrupted by a crash or quota limit

The problemLong model-backed runs cost money, and restarting from scratch after an interruption wastes both time and API spend.

The outcomeA completed run that picked up where it stopped, with results kept under the same runs directory.

What you need first

• Linux, Python 3.12 and uv for installation
• No credentials needed for the local demo (harness2-demo)
• Model-backed runs need Node.js 20+, the Google Cloud CLI, a Vertex AI project and model IDs in .env, and gcloud application-default login
• LAB needs Pandoc and Poppler; WorkBuddy-Bench needs Docker
• Model-backed benchmark runs incur API costs

Steps

1. Note the exact command you used, including --bench, --k, --mode, any subset flags and seed.
2. Re-run the identical harness2 command to resume.
3. Use a new HARNESS2_RUNS_DIR only if you changed models or task selection.
4. Run harness2-judge and harness2-report with the same flags once refinement finishes.
5. Check each command's --help for extra options.

Prompt for your agent

My google-research/harness2 run on <benchmark> stopped partway. Reconstruct the exact original command, re-run it to resume, and verify that previously finished tasks were not repeated by checking the runs/ folder.

Caveat. Resume only works when you repeat the same benchmark, executor, subset, --k, --mode and seed, and changing models or task selection requires a new HARNESS2_RUNS_DIR.

• Markdown version
• JSON version