# AI use cases for Loop Arena (AMAP-ML/LoopArena)

Repo: [AMAP-ML/LoopArena](https://github.com/AMAP-ML/LoopArena) | Use cases: 3 | Page: https://dedupedai.com/use-cases/amap-ml-looparena/

LoopArena is a Python benchmark that tests how well AI models work as runtime Controllers that guide a separate coding Worker through a Loop Engineering task.

This file holds every use case of this repo. Each one starts with a "## " heading and ends with its URL.

- Check a Controller model on Type I questions: https://dedupedai.com/use-cases/amap-ml-looparena/smoke-test-controller-model-type1/
- Compare Controller models on paired coding tasks: https://dedupedai.com/use-cases/amap-ml-looparena/compare-controller-models-coding-tasks/
- Pick a Controller model from published results: https://dedupedai.com/use-cases/amap-ml-looparena/pick-controller-from-published-results/

## How to check a Controller model on Type I questions with Loop Arena

Repo: [AMAP-ML/LoopArena](https://github.com/AMAP-ML/LoopArena) | Difficulty: Intermediate | Time: 30 minutes | Basis: From the README

Run one model on the 160 Type I Loop Contract questions without a Worker.

**Who it is for:** An AI engineer who wants to see whether a candidate model can pick the right Loop Contract before a longer benchmark run.

**The problem:** Type II and III need Docker, task data, and many runs. A quick Type I check tells you if the model can answer the basic Controller decision.

**The outcome:** A JSONL output file with the model's answer to a single Type I question, after a preflight check of the endpoint and configuration.

### Steps

1. Install Python 3.10+ and run `python -m pip install -e '.[gateway]'` from the repo root.
2. Set your endpoint details with `export OPENAI_API_KEY=YOUR_API_KEY` and `export OPENAI_BASE_URL=YOUR_BASE_URL`, or copy the values from `.env.example`.
3. Run `looparena-type1-run --data benchmarks/type1/questions --model MODEL_ID --preflight-only` with your model ID in place of MODEL_ID. (inferred, check the README)
4. Run `looparena-type1-run --data benchmarks/type1/questions --model MODEL_ID --output runs/type1-smoke.jsonl --limit 1`. (inferred, check the README)
5. Open runs/type1-smoke.jsonl and confirm it contains one result for the chosen model.

### Prompt to paste into your agent

```text
In the AMAP-ML/LoopArena repo, set OPENAI_API_KEY and OPENAI_BASE_URL from the environment, then run looparena-type1-run with --data benchmarks/type1/questions, --model <model ID>, and --preflight-only, followed by a one-question run with --limit 1 writing to runs/type1-smoke.jsonl. Report whether preflight passed and the path of the output file, then confirm the file holds exactly one result.
```

### Caveat

The README gives the model ID only as a placeholder, so you must supply a model your endpoint actually serves.

Risk, credentials: The setup uses an API key set in environment variables, so keep it out of committed files and shared logs.

Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/

URL: https://dedupedai.com/use-cases/amap-ml-looparena/smoke-test-controller-model-type1/

## How to compare Controller models on paired coding tasks with Loop Arena

Repo: [AMAP-ML/LoopArena](https://github.com/AMAP-ML/LoopArena) | Difficulty: Advanced | Time: 4 hours | Basis: From the README

Run candidate Controllers on Type II and III coding tasks with a shared Worker.

**Who it is for:** A coding-agent team that must decide which model should act as the runtime Controller for long repository edits.

**The problem:** Single-answer tests do not show whether a Controller can keep a Worker on track, verify progress, and stop at the right time across a full coding task.

**The outcome:** Strict Success Rate results for each candidate Controller across the 80 paired tasks, each repeated 3 times, using source-native terminal evaluation.

### Steps

1. Read docs/protocol.md to confirm the role boundaries, budgets, and scoring.
2. Follow docs/setup.md to pin the upstream source revisions, prepare the task data and starting workspaces, and install the Docker runtimes.
3. Set the Controller model and the shared Worker and Reporter model in the configuration described in docs/setup.md. (inferred, check the README)
4. Run a single case first, then the full panel, using the single-case and panel run instructions in docs/setup.md.
5. Compare the Strict Success Rate for each Controller against the Core-based scores in the README results table.

### Prompt to paste into your agent

```text
In the AMAP-ML/LoopArena repo, follow docs/setup.md to prepare the Type II coding task data and Docker runtimes, then run one single-case Type II test with a <Controller model ID> as Controller and the shared Worker and Reporter from the setup guide. Report the Strict Success Rate for that case and the path of the run output, and verify that the protocol and budget validity checks passed.
```

### Caveat

The README does not give setup times or resource needs, and the upstream repositories and container images must be obtained from their original sources, so the time estimate is rough.

Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/

URL: https://dedupedai.com/use-cases/amap-ml-looparena/compare-controller-models-coding-tasks/

## How to pick a Controller model from published results with Loop Arena

Repo: [AMAP-ML/LoopArena](https://github.com/AMAP-ML/LoopArena) | Difficulty: Beginner | Time: 15 minutes | Basis: From the README

Compare six Controller models on Type I, II, and III before running anything.

**Who it is for:** A technical lead choosing which model to test or deploy as a coding Controller and needing a quick, cost-aware comparison.

**The problem:** Running the benchmark takes setup and budget. The published leaderboard and data show how the models compare first.

**The outcome:** A short shortlist of Controller models based on the published Type I accuracy and Type II/III Strict Success Rate, with the estimated cost in view.

### Steps

1. Open the README results table and note the Type I accuracy, Type II SSR, and Type III SSR for each of the six Controller models.
2. Open the interactive leaderboard at https://amap-ml.github.io/LoopArena/#results to see reference policies and estimated costs.
3. Open docs/data/results.json to check the aggregate data for all three settings.
4. Shortlist the models that score well on Type II and III, and note that costs are normalized no-cache inference estimates, not invoices.

### Prompt to paste into your agent

```text
Using the AMAP-ML/LoopArena README results table and docs/data/results.json, rank the six Controller models by Type III Strict Success Rate and list their Type I accuracy and estimated cost side by side. Report the ranking as a table and note that the costs are normalized estimates, not invoices.
```

### Caveat

The published scores use the primary Core-based metric and a shared Qwen3.7-Plus Worker, so they may not match your own Worker setup.

Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/

URL: https://dedupedai.com/use-cases/amap-ml-looparena/pick-controller-from-published-results/