# AI use cases for Harness Zero (metaevo-ai/harness-zero) Repo: [metaevo-ai/harness-zero](https://github.com/metaevo-ai/harness-zero) | Use cases: 4 | Page: https://dedupedai.com/use-cases/metaevo-ai-harness-zero/ Harness-Zero is a Python research repo that distills an optimized agent harness into Qwen3.5-9B model weights, so the gains remain when the harness is removed at deployment. This file holds every use case of this repo. Each one starts with a "## " heading and ends with its URL. - Generate reviewed agent trajectories for training data: https://dedupedai.com/use-cases/metaevo-ai-harness-zero/generate-reviewed-agent-trajectories/ - Distill a harness into Qwen3.5-9B weights: https://dedupedai.com/use-cases/metaevo-ai-harness-zero/distill-harness-into-model-weights/ - Adapt the harness bank to a new task domain: https://dedupedai.com/use-cases/metaevo-ai-harness-zero/adapt-harness-bank-to-new-domain/ - Reproduce the AppWorld evaluation with the dedicated setup: https://dedupedai.com/use-cases/metaevo-ai-harness-zero/reproduce-appworld-evaluation/ ## How to generate reviewed agent trajectories for training data with Harness Zero Repo: [metaevo-ai/harness-zero](https://github.com/metaevo-ai/harness-zero) | Difficulty: Advanced | Time: 2 hours | Basis: From the README Run a harnessing agent over benchmark tasks and export successful trials as an SFT dataset. **Who it is for:** An ML researcher who wants training trajectories for a small student model and has Docker and a model API key ready. **The problem:** Collecting agent trajectories by hand is slow, and raw runs without review do not carry the behaviors of an optimized harness. **The outcome:** A reviewed rollout directory of trial results and an sft.jsonl file built from trials that met the reward threshold. ### Steps 1. Run `uv sync` in the cloned Harness Zero folder. 2. Copy .env.example to .env and fill in the key for the model route you use. 3. Create a task list with `printf 'uspto-train-000\nuspto-train-001\n' > /tmp/tasks.txt`. 4. Run `harness-zero run-rollout --dataset data/uspto --components harness_bank/uspto --teacher-middleware-factory harness_bank.uspto.middlewares:build_teacher_middlewares --tasks-file /tmp/tasks.txt --attempts 1 --concurrency 8 --student-model openai: --student-reasoning-effort high --teacher-provider openai --teacher-model --teacher-reasoning-effort high --job-name demo --output-dir runs`, filling in the model names for your route. (inferred, check the README) 5. Run `harness-zero build-sft --trials runs/demo --output sft.jsonl --reward-threshold 1.0`. ### Prompt to paste into your agent ```text Using the repo metaevo-ai/harness-zero, run a reviewed rollout on the task IDs in with the USPTO harness bank, then build an SFT dataset from runs/demo. Expected output: sft.jsonl containing only trials that met reward threshold 1.0. Verify that runs/demo contains trial results before you run build-sft. ``` ### Caveat The README names the USPTO route in the quickstart; other domains need their own harness_bank folder and task data. Risk, credentials: The .env file holds live model API keys, so keep it out of version control and do not paste keys into agent prompts. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/metaevo-ai-harness-zero/generate-reviewed-agent-trajectories/ ## How to distill a harness into Qwen3.5-9B weights with Harness Zero Repo: [metaevo-ai/harness-zero](https://github.com/metaevo-ai/harness-zero) | Difficulty: Advanced | Time: 3 hours | Basis: From the README Fine-tune the base model on reviewed trajectories so it runs well under the minimal harness alone. **Who it is for:** An agent engineer who wants a deployable model that keeps the gains of an optimized harness without shipping the harness itself. **The problem:** An optimized harness lifts performance only while it is attached, which makes deployment depend on extra infrastructure. **The outcome:** A LoRA-tuned Qwen3.5-9B checkpoint trained through the Tinker recipe, ready to run under the minimal mini-SWE-agent-style harness. ### Steps 1. Run `uv sync --group tinker` to add the training dependencies. 2. Make sure sft.jsonl exists from a reviewed rollout and build step. 3. Run `harness-zero train-sft --data sft.jsonl --base-model Qwen/Qwen3.5-9B --renderer qwen3_5 --rank 32 --peak-learning-rate 2e-4 --final-learning-rate 1e-6 --warmup-ratio 0.05 --epochs 2 --batch-size 8 --max-length 65536 --seed 42 --run-name hz-demo --output-dir runs/train`. (inferred, check the README) 4. Confirm the submission prompt after checking the run settings, or pass `--yes` to skip the prompt. ### Prompt to paste into your agent ```text Using the repo metaevo-ai/harness-zero, train a LoRA SFT run from sft.jsonl on Qwen/Qwen3.5-9B with run name hz-demo. Expected output: a training run under runs/train. Verify the confirmation prompt shows the intended dataset and settings before you approve the submission. ``` ### Caveat Training runs through Tinker, and the README does not describe its costs, so confirm the job before you approve it. Risk, credentials: Training uses a Tinker key (TM_API_KEY), so store it in .env only and avoid sharing it with agents. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/metaevo-ai-harness-zero/distill-harness-into-model-weights/ ## How to adapt the harness bank to a new task domain with Harness Zero Repo: [metaevo-ai/harness-zero](https://github.com/metaevo-ai/harness-zero) | Difficulty: Advanced | Time: 3 hours | Basis: From the README Use the bundled evolution skills to build student and reference harnesses for another domain. **Who it is for:** A researcher who has a new benchmark task set and wants to apply the same harness distillation method. **The problem:** Each domain needs its own student harness and reference harness, and the method is hard to start without a worked pattern. **The outcome:** A new folder in harness_bank with an evolved student-side harness and an adapted reference harness, modeled on the existing domains. ### Steps 1. Run `uv sync` and read the repository layout section of the README. 2. Open the existing folders under harness_bank/ and compare the *_student harness with the reference harness. 3. Read the skills in .agents/skills/ that were used to evolve and adapt the harnesses. 4. Create a new folder under harness_bank/ for your domain and place the task set under data/ in the same Harbor format used by the three benchmarks. (inferred, check the README) ### Prompt to paste into your agent ```text Using the repo metaevo-ai/harness-zero, study harness_bank/uspto and the skills in .agents/skills/, then draft a new harness_bank folder for the domain . Expected output: a folder layout and a short plan listing the student harness and reference harness files. Verify that the plan matches the existing per-domain structure before writing any files. ``` ### Caveat The README describes the evolution skills only at a high level, so the new domain setup will require reading the skill files closely. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/metaevo-ai-harness-zero/adapt-harness-bank-to-new-domain/ ## How to reproduce the AppWorld evaluation with the dedicated setup with Harness Zero Repo: [metaevo-ai/harness-zero](https://github.com/metaevo-ai/harness-zero) | Difficulty: Advanced | Time: 4 hours | Basis: From the README Run AppWorld rollouts through the included mini-SWE-agent and environment to match the reported setup. **Who it is for:** A benchmark researcher who needs to check AppWorld results against the paper's test_normal split. **The problem:** AppWorld rollouts fail or give misleading scores when run with a generic harness instead of the dedicated agent and environment. **The outcome:** AppWorld rollouts that use the dedicated envs/appworld agent and environment, with SGC scores on the 168 test tasks. ### Steps 1. Run `uv sync` in the repo folder. 2. Read envs/appworld/README.md, which the README says AppWorld rollouts must follow. 3. Build the AppWorld image and environment as described in envs/appworld/README.md. 4. Run the AppWorld rollouts with the dedicated mini-SWE-agent and environment, then compare SGC against the test_normal numbers in the README. ### Prompt to paste into your agent ```text Using the repo metaevo-ai/harness-zero, follow envs/appworld/README.md to run AppWorld test_normal rollouts with the dedicated agent and environment. Expected output: SGC scores across the 168 test tasks. Verify that the run uses the dedicated environment and not a generic harness before reporting scores. ``` ### Caveat The README says AppWorld rollouts must use the dedicated environment, so results from other setups are not comparable to the paper. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/metaevo-ai-harness-zero/reproduce-appworld-evaluation/