DEDUPEDWho it is forA model developer who has a trained checkpoint and needs honest metrics on the validation or test split before deciding to use it.
The problemA single mAP number hides where the model fails, and rerunning evaluation by hand loses context.
The outcomeSaved metrics, logs, confusion matrix, PR curves when generated, and a list of missed detections, false positives, and low-confidence samples to review.
What you need first
- macOS or Linux
- Python 3.11 or 3.12
- Node.js 20 or newer
- A set of images to label, or the built-in tiny demo generator
Steps
- Start the workbench with
./run-all.sh. - Open the model workspace and pick a managed PT model produced by a training run.
- Create an evaluation task on the train, val, or test split and wait for the background job to finish.
- Open the evaluation details to read the metrics, confusion matrix, and charts, then review the error samples.
- Compare results across runs only on the same dataset, since the README notes that a data snapshot fingerprint is not yet verifiable.
Prompt for your agent
In the YoloWebAgentStarter repo at <repo path>, run a background evaluation of managed model <model ID> on the test split of dataset <dataset ID>. Verify that the metrics, logs, and error sample list are saved and at most 200 entries. Return the box metrics and the count of each error type.
Caveat. Chart files only appear when Ultralytics generates them, so a tiny random model may produce metrics and logs without any PR curve.
Use this in your AI agent
Ask your agent
Using the Deduped MCP, get the setup steps for fengzhizi715/YoloWebAgentStarter and "Score a trained model on a held-out split and inspect its mistakes"
Add Deduped to Claude Code once
claude mcp add --transport http deduped https://dedupedai.com/mcp
Every Monday: the 5 hottest repos and exactly how to use them. Free, one email a week, unsubscribe anytime.
Get them freeShare: X LinkedIn Hacker News Email Copy link
Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer