DEDUPED Hot AI repos, turned into use cases Get 5 a day

What you can do with Doc2 Harness

Doc2Harness is a Python benchmark that measures how well a model builds a working coding agent harness, then tests that harness against a scripted mock model using 94 released scenarios.

Trended earlier

YuyaoGe/doc2harness on GitHub

Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk.

What do you want to do?

4 use cases

  • Score an existing coding agent harness

    Run all 94 released scenarios against your product directory and get a pass rate.

    View prompt
    Use the repo YuyaoGe/doc2harness to score the product at <path-to-product-directory>. Run tools/run_production_catalog.py with --artifact <path-to-product-directory> and --output <output-dir>, then report the pass and fail counts from catalog_receipt.json. Verify that the product directory contains src/yuyao/ before running.

  • Re-run one scenario family to debug a harness

    Narrow scoring to one family, such as headless or permission, to find a failing area.

    View prompt
    Use the repo YuyaoGe/doc2harness to re-run only the <family-name> scenario family against the product at <path-to-product-directory>. Use tools/run_production_catalog.py with --family <family-name> and report which scenarios failed. Verify the catalog still loads with the family filter applied.

  • Confirm the scenario catalog is intact before scoring

    Verify that the catalog loads and contains 94 scenarios before any scoring run.

    View prompt
    In the repo YuyaoGe/doc2harness, run the Python sanity check that imports active_scenario_ids and prints its length. Report the printed number and any digest error. Verify that the output is exactly 94 before reporting the catalog as intact.

  • Build a new agent harness and score it

    Use the builder image and your own model key to generate a harness, then score it.

    View prompt
    Use the repo YuyaoGe/doc2harness to build a new harness with the builder tag <builder-tag> and output to <output-dir>, using the local LiteLLM gateway at <gateway-url>. Run the dry-run first, then the real build, and report the output path. Verify the dry-run printed the full plan before starting the real build.

How to use Doc2 Harness

  1. Check you have what it needs: Python 3.11 or newer; Docker installed and running; The repo cloned locally.
  2. Open YuyaoGe/doc2harness on GitHub and read the README and license.
  3. Run docker build --target evaluation -t doc2harness/evaluation:candidate -f containers/Dockerfile . from the repo root (see containers/README.md for mirror options).
  4. Confirm your product directory contains src/yuyao/, then run PYTHONPATH=src:tools:run python3 tools/run_production_catalog.py --artifact /path/to/product --artifact-id my-product --output /tmp/scores --concurrency 12 --timeout 300. (inferred, check the README)
  5. Wait about 25 to 90 seconds at concurrency 12 for the full catalog to finish.
  6. Open /tmp/scores/catalog_receipt.json and read the score, remembering that INCONCLUSIVE is not counted as a failure.

These steps come from our top use case, Score an existing coding agent harness, which also has a copy-paste agent prompt.

Doc2 Harness alternatives · Best AI repos for Software and DevOps

About this repo

Requirements

  • Python 3.11 or newer
  • Docker installed and running
  • The repo cloned locally
  • For building a new harness only: the builder image, a local LiteLLM gateway, and your own API key
Maintain Doc2 Harness? Add the Featured on Deduped badge

Paste this into your README:

[![Featured on Deduped](https://dedupedai.com/badge/yuyaoge-doc2harness.svg)](https://dedupedai.com/use-cases/yuyaoge-doc2harness/?ref=badge)

More badge styles and HTML

Questions about Doc2 Harness

What is Doc2 Harness?

Doc2Harness is a Python benchmark that measures how well a model builds a working coding agent harness, then tests that harness against a scripted mock model using 94 released scenarios. It is an open-source project on GitHub: https://github.com/YuyaoGe/doc2harness.

Who is Doc2 Harness for?

Coding agent developers and AI researchers who need to score a harness they built or a harness a model generated.

What can I do with Doc2 Harness?

4 use cases so far, for example: Score an existing coding agent harness; Re-run one scenario family to debug a harness; Confirm the scenario catalog is intact before scoring. Each has steps and a copy-paste agent prompt.

What do I need to run Doc2 Harness?

Python 3.11 or newer; Docker installed and running; The repo cloned locally; For building a new harness only: the builder image, a local LiteLLM gateway, and your own API key.

What are alternatives to Doc2 Harness?

Other hot AI repos in Software and DevOps: Harness2, iFixAi, Orca, Diagram Design, AI Memory.

Is Doc2 Harness safe to run?

Deduped does not own, audit or vouch for it. Read the README and license, run it on your own systems first and review every command before you run it.