DEDUPED Hot AI repos, turned into use cases Get the Monday 5

How to pick a Controller model from published results with Loop Arena

Compare six Controller models on Type I, II, and III before running anything.

Short answer: Open the README results table and note the Type I accuracy, Type II SSR, and Type III SSR for each of the six Controller models; then open the interactive leaderboard at https://amap-ml.github.io/LoopArena/#results to see reference policies and estimated costs; open docs/data/results.json to check the aggregate data for all three settings. All steps.

  • Research
  • Beginner
  • 15 minutes
  • From the README
  • Any agent

Who it is forA technical lead choosing which model to test or deploy as a coding Controller and needing a quick, cost-aware comparison.

The problemRunning the benchmark takes setup and budget. The published leaderboard and data show how the models compare first.

The outcomeA short shortlist of Controller models based on the published Type I accuracy and Type II/III Strict Success Rate, with the estimated cost in view.

What you need first

  • Python 3.10 or newer
  • An OpenAI-compatible model endpoint, its base URL, and an API key
  • The model ID you want to test
  • For Type II/III only: the pinned upstream task data, starting workspaces, and Docker runtimes described in docs/setup.md

Steps

  1. Open the README results table and note the Type I accuracy, Type II SSR, and Type III SSR for each of the six Controller models.
  2. Open the interactive leaderboard at https://amap-ml.github.io/LoopArena/#results to see reference policies and estimated costs.
  3. Open docs/data/results.json to check the aggregate data for all three settings.
  4. Shortlist the models that score well on Type II and III, and note that costs are normalized no-cache inference estimates, not invoices.

Prompt for your agent

Using the AMAP-ML/LoopArena README results table and docs/data/results.json, rank the six Controller models by Type III Strict Success Rate and list their Type I accuracy and estimated cost side by side. Report the ranking as a table and note that the costs are normalized estimates, not invoices.

Caveat. The published scores use the primary Core-based metric and a shared Qwen3.7-Plus Worker, so they may not match your own Worker setup.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for AMAP-ML/LoopArena and "Pick a Controller model from published results"

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer