DEDUPED Hot AI repos, turned into use cases Get the Monday 5
164 stars · JavaScript

How to use Bug Hunt Bench: what you can do with it

What is Bug Hunt Bench? A benchmark of 105 real bugs in two production codebases that grades frontier AI coding models, each run in its own agentic CLI, with blind grading against a withheld answer key.

How do I use it? Open the live board at bughunt.productcompass.pm; then read the Fixed /105 column and the Repo 1 /45 and Repo 2 /60 columns to see where each model does well; Compare the Cost column for the models that score within a few points of the top. Full steps below.

Trended earlier

View repo on GitHub

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer

What do you want to do?

4 use cases

  • Choose a coding model for bug fixing by score and cost

    Rank models by planted bugs fixed, per repo and per dollar, before you pick one for fixes.

    View prompt
    Using the bug-hunt-bench leaderboard at bughunt.productcompass.pm, list the top five models by Fixed /105 along with their Repo 1 and Repo 2 scores and Cost for <task or budget note>. Output a short table. Verify that each score matches the row in the README table before reporting it.

  • Compare one model across different coding CLIs

    See whether the tool wrapped around a model changes its bug-fixing score.

    View prompt
    Using the bug-hunt-bench README table, list every row for <model name> with its Harness, Effort, and Fixed /105 score, sorted by score. Output a table. Verify that each row's harness and effort match the source table.

  • Study the 25 bugs no model has fixed

    Look at the bugs that every model missed to learn where current tools fail.

    View prompt
    In the bug-hunt-bench repo, open results/waves.md and the raw data in results/, then list the bugs that no model fixed, grouped by repo. Output a short report. Verify that the count of unfixed bugs matches the 25 of 105 stated in the README.

  • Audit a model's bug-fix claims against the receipts

    Check a leaderboard score against the raw run data before you trust it.

    View prompt
    In the bug-hunt-bench repo, open the results/ folder and find the raw data for <model name>. Report its run count, Fixed /105 score, and per-repo scores, and state whether they match the README leaderboard row. Verify the totals before you report.

Quick start with Bug Hunt Bench

  1. Check you have what it needs: A web browser to open the live leaderboard at bughunt.productcompass.pm; Access to the results/ folder on GitHub or a local copy of the repo for the raw data.
  2. Open phuryn/bug-hunt-bench on GitHub and read the README and license.
  3. Open the live board at bughunt.productcompass.pm.
  4. Read the Fixed /105 column and the Repo 1 /45 and Repo 2 /60 columns to see where each model does well.
  5. Compare the Cost column for the models that score within a few points of the top.
  6. Open the Method page at bughunt.productcompass.pm/method and read the definitions and caveats before you decide.

These steps come from our top use case, Choose a coding model for bug fixing by score and cost, which also has a copy-paste agent prompt.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for phuryn/bug-hunt-bench

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Bug Hunt Bench alternatives · Best AI repos for Software and DevOps

About this repo

  • Who it is forWritten for engineering leads, researchers, and agents who need to compare AI coding models on bug fixing before spending money or time on them.
  • Why it is hot164 stars, launched 62 days ago
  • LanguageJavaScript
  • Sourcegithub.com/phuryn/bug-hunt-bench · Back to Hot repos

Requirements

  • A web browser to open the live leaderboard at bughunt.productcompass.pm
  • Access to the results/ folder on GitHub or a local copy of the repo for the raw data

Add this badge to your README

Maintain Bug Hunt Bench? Show readers where to find these use cases. Use cases on Deduped

Markdown 

[![Use cases on Deduped](https://dedupedai.com/badge/phuryn/bug-hunt-bench.svg)](https://dedupedai.com/use-cases/phuryn-bug-hunt-bench/?utm_source=github&utm_medium=badge&utm_campaign=phuryn-bug-hunt-bench)

HTML 

<a href="https://dedupedai.com/use-cases/phuryn-bug-hunt-bench/?utm_source=github&amp;utm_medium=badge&amp;utm_campaign=phuryn-bug-hunt-bench"><img src="https://dedupedai.com/badge/phuryn/bug-hunt-bench.svg" alt="Use cases on Deduped" height="20"></a>

A static image, no tracking script. About the badge

Questions about Bug Hunt Bench

What is Bug Hunt Bench?

A benchmark of 105 real bugs in two production codebases that grades frontier AI coding models, each run in its own agentic CLI, with blind grading against a withheld answer key. It is an open-source project on GitHub: https://github.com/phuryn/bug-hunt-bench.

How do I use Bug Hunt Bench?

Open the live board at bughunt.productcompass.pm; then read the Fixed /105 column and the Repo 1 /45 and Repo 2 /60 columns to see where each model does well; Compare the Cost column for the models that score within a few points of the top. The use cases on this page give the full steps and a copy-paste agent prompt for each job.

Who is Bug Hunt Bench for?

Written for engineering leads, researchers, and agents who need to compare AI coding models on bug fixing before spending money or time on them.

What can I do with Bug Hunt Bench?

4 use cases so far, for example: Choose a coding model for bug fixing by score and cost; Compare one model across different coding CLIs; Study the 25 bugs no model has fixed. Each has steps and a copy-paste agent prompt.

What do I need to run Bug Hunt Bench?

A web browser to open the live leaderboard at bughunt.productcompass.pm; Access to the results/ folder on GitHub or a local copy of the repo for the raw data.

What are alternatives to Bug Hunt Bench?

Other hot AI repos in Software and DevOps: Deep Infer, Harness2, Orca, Opencode, Graphify.

Is Bug Hunt Bench safe to run?

Deduped does not own, audit or vouch for it. Read the README and license, run it on your own systems first and review every command before you run it.