DEDUPED Hot AI repos, turned into use cases Get 5 a day

How to compare recursion budgets on WorkBuddy-Bench code tasks with Harness2

Run parallel and sequential refinement on the WorkBuddy code subset to see which recursion style helps more.

Who it is forAn AI agent researcher who needs to choose a recursion strategy for coding-style office tasks and wants a fair side-by-side comparison.

The problemParallel and sequential recursion trade breadth for depth, and the right choice depends on the benchmark and budget.

The outcomeSeparate judged and reported runs for each recursion mode on the WorkBuddy code subset, using a distinct runs directory for each.

What you need first

  • Linux, Python 3.12, and uv
  • For the local demo: nothing else, since it needs no model credentials
  • For model-backed runs: Node.js 20+, the Google Cloud CLI, a Vertex AI project with model IDs, and API budget
  • For LAB: Pandoc and Poppler. For WorkBuddy-Bench: Docker

Steps

  1. Set up the environment with uv venv --python 3.12 .venv, source .venv/bin/activate, and uv pip install -e '.[render]', then run harness2-setup wb (Docker must be installed)
  2. Check the install with harness2-check --bench wb (inferred, check the README)
  3. Run harness2 --bench wb --subset code --k 2 --mode parallel with HARNESS2_RUNS_DIR set to a new folder for this mode (inferred, check the README)
  4. Run harness2 --bench wb --subset code --k 2 --mode sequential with a different HARNESS2_RUNS_DIR (inferred, check the README)
  5. Judge and report each mode with harness2-judge --bench wb --subset code --k 2 --mode <mode> and harness2-report --bench wb --subset code --k 2 --mode <mode> (inferred, check the README)
  6. Compare the two reports, keeping the same benchmark, subset, k, and seed for both

Prompt for your agent

In the google-research/harness2 repo, install WorkBuddy-Bench with harness2-setup wb, then run the code subset in parallel and sequential mode with --k 2, using a separate HARNESS2_RUNS_DIR for each. Report the judged score for each mode and which one scored higher. Verify that both runs used the same subset, k, and seed.

Caveat. The README says the executor has unrestricted network access and Codex runs with its sandbox disabled, so configure the network restrictions in the experiment guide before evaluation.

Share: X LinkedIn Hacker News Email Copy link

Disclaimer

Deduped lists open-source repositories that belong to other people. We do not own, maintain, audit, or vouch for any of them, and a repo appearing here is not an endorsement or a safety check. The use cases, steps, and prompts are written by AI and may be incomplete, outdated, or just wrong. Where a use case is marked "Seen in the wild", we link to public posts where someone described something similar, but we have not verified those posts and the use case itself is still our AI's own write-up. Use all of this at your own risk. Only run code, commands, or agents on systems and accounts you are authorized to use, and read the code and every command before you run it, especially anything that touches credentials, files, or the network. Nothing here is professional, security, legal, or financial advice. All trademarks and project names belong to their respective owners, and their use here does not imply affiliation. Content is provided as is, without warranties of any kind. Found a mistake, a safety problem, or something you want removed? Email [email protected] and we will look at it promptly. Full disclaimer page