DEDUPED Hot AI repos, turned into use cases Get the Monday 5

How to improve an agent version by version with benchmarks with Adelie

Run a benchmark, find the drop in score, and ship version N+1 with replayable traces.

Short answer: Install and start Adelie with npm install -g @adelie/cli and then penguin web; then open the agent you want to improve and run a Benchmark from the interface; open the trace viewer and find the requests where the score dropped. All steps.

Who it is forAn AI engineer whose agent scores lower after a change and needs to find which step caused the regression.

The problemAgent quality drifts between versions and it is hard to see which request or round caused the score to fall.

The outcomeThe agent is evaluated on a Benchmark, the weak area is identified, and a new version is produced with a snapshot taken before each round and every request replayable in the trace viewer.

What you need first

  • A computer running Windows 10+ (x64), Linux (x64), or macOS (arm64 or x64)
  • Node 24 or newer if you use the npm or from-source install
  • An API key from at least one supported model provider, entered on the Models page
  • For browser tasks: credentials for the sites you want the agent to use, imported from your own browser

Steps

  1. Install and start Adelie with npm install -g @adelie/cli and then penguin web.
  2. Open the agent you want to improve and run a Benchmark from the interface.
  3. Open the trace viewer and find the requests where the score dropped.
  4. Note the snapshot taken before the round, then produce version N+1 of the agent through the self-evolution engine.
  5. Run the Benchmark again on version N+1 and compare the results with the previous round.

Prompt for your agent

Using the Adelie repo (lmliheng/Adelie), run a Benchmark on the agent <agent name> in the local web interface, locate the requests whose score fell in the trace viewer, and produce version N+1. Expected output: a new agent version with a Benchmark score for each round. Verify that the snapshot from before the round can be restored.

Caveat. The README shows the benchmark chart from upstream PenguinHarness measurements and the upstream roadmap lists the benchmark suite as not yet released, so check what the Benchmark feature covers in your build before relying on it.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for lmliheng/Adelie and "Improve an agent version by version with benchmarks"

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer