DEDUPED Hot AI repos, turned into use cases Get the Monday 5
152 stars · Python

How to use Loop Arena: what you can do with it

What is Loop Arena? LoopArena is a Python benchmark that tests how well AI models work as runtime Controllers that guide a separate coding Worker through a Loop Engineering task.

How do I use it? Install Python 3.10+ and run python -m pip install -e '.[gateway]' from the repo root; then set your endpoint details with export OPENAI_API_KEY=YOUR_API_KEY and export OPENAI_BASE_URL=YOUR_BASE_URL, or copy the values from .env.example; run looparena-type1-run --data benchmarks/type1/questions --model MODEL_ID --preflight-only with your model ID in place of MODEL_ID. Full steps below.

Trended earlier

View repo on GitHub

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer

What do you want to do?

3 use cases

  • Check a Controller model on Type I questions

    Run one model on the 160 Type I Loop Contract questions without a Worker.

    View prompt
    In the AMAP-ML/LoopArena repo, set OPENAI_API_KEY and OPENAI_BASE_URL from the environment, then run looparena-type1-run with --data benchmarks/type1/questions, --model <model ID>, and --preflight-only, followed by a one-question run with --limit 1 writing to runs/type1-smoke.jsonl. Report whether preflight passed and the path of the output file, then confirm the file holds exactly one result.

  • Compare Controller models on paired coding tasks

    Run candidate Controllers on Type II and III coding tasks with a shared Worker.

    View prompt
    In the AMAP-ML/LoopArena repo, follow docs/setup.md to prepare the Type II coding task data and Docker runtimes, then run one single-case Type II test with a <Controller model ID> as Controller and the shared Worker and Reporter from the setup guide. Report the Strict Success Rate for that case and the path of the run output, and verify that the protocol and budget validity checks passed.

  • Pick a Controller model from published results

    Compare six Controller models on Type I, II, and III before running anything.

    View prompt
    Using the AMAP-ML/LoopArena README results table and docs/data/results.json, rank the six Controller models by Type III Strict Success Rate and list their Type I accuracy and estimated cost side by side. Report the ranking as a table and note that the costs are normalized estimates, not invoices.

Quick start with Loop Arena

  1. Check you have what it needs: Python 3.10 or newer; An OpenAI-compatible model endpoint, its base URL, and an API key; The model ID you want to test.
  2. Open AMAP-ML/LoopArena on GitHub and read the README and license.
  3. Install Python 3.10+ and run python -m pip install -e '.[gateway]' from the repo root.
  4. Set your endpoint details with export OPENAI_API_KEY=YOUR_API_KEY and export OPENAI_BASE_URL=YOUR_BASE_URL, or copy the values from .env.example.
  5. Run looparena-type1-run --data benchmarks/type1/questions --model MODEL_ID --preflight-only with your model ID in place of MODEL_ID. (inferred, check the README)
  6. Run looparena-type1-run --data benchmarks/type1/questions --model MODEL_ID --output runs/type1-smoke.jsonl --limit 1. (inferred, check the README)

These steps come from our top use case, Check a Controller model on Type I questions, which also has a copy-paste agent prompt.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for AMAP-ML/LoopArena

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Loop Arena alternatives · Best AI repos for Research

About this repo

  • Who it is forWritten for AI researchers and engineers who need to compare models as Controllers, and for agents asked to run the benchmark.
  • Why it is hot152 stars, launched 56 days ago
  • LanguagePython
  • Sourcegithub.com/AMAP-ML/LoopArena · Back to Hot repos

Requirements

  • Python 3.10 or newer
  • An OpenAI-compatible model endpoint, its base URL, and an API key
  • The model ID you want to test
  • For Type II/III only: the pinned upstream task data, starting workspaces, and Docker runtimes described in docs/setup.md

Add this badge to your README

Maintain Loop Arena? Show readers where to find these use cases. Use cases on Deduped

Markdown 

[![Use cases on Deduped](https://dedupedai.com/badge/AMAP-ML/LoopArena.svg)](https://dedupedai.com/use-cases/amap-ml-looparena/?utm_source=github&utm_medium=badge&utm_campaign=AMAP-ML-LoopArena)

HTML 

<a href="https://dedupedai.com/use-cases/amap-ml-looparena/?utm_source=github&amp;utm_medium=badge&amp;utm_campaign=AMAP-ML-LoopArena"><img src="https://dedupedai.com/badge/AMAP-ML/LoopArena.svg" alt="Use cases on Deduped" height="20"></a>

A static image, no tracking script. About the badge

Questions about Loop Arena

What is Loop Arena?

LoopArena is a Python benchmark that tests how well AI models work as runtime Controllers that guide a separate coding Worker through a Loop Engineering task. It is an open-source project on GitHub: https://github.com/AMAP-ML/LoopArena.

How do I use Loop Arena?

Install Python 3.10+ and run python -m pip install -e '.[gateway]' from the repo root; then set your endpoint details with export OPENAI_API_KEY=YOUR_API_KEY and export OPENAI_BASE_URL=YOUR_BASE_URL, or copy the values from .env.example; run looparena-type1-run --data benchmarks/type1/questions --model MODEL_ID --preflight-only with your model ID in place of MODEL_ID. The use cases on this page give the full steps and a copy-paste agent prompt for each job.

Who is Loop Arena for?

Written for AI researchers and engineers who need to compare models as Controllers, and for agents asked to run the benchmark.

What can I do with Loop Arena?

3 use cases so far, for example: Check a Controller model on Type I questions; Compare Controller models on paired coding tasks; Pick a Controller model from published results. Each has steps and a copy-paste agent prompt.

What do I need to run Loop Arena?

Python 3.10 or newer; An OpenAI-compatible model endpoint, its base URL, and an API key; The model ID you want to test; For Type II/III only: the pinned upstream task data, starting workspaces, and Docker runtimes described in docs/setup.md.

What are alternatives to Loop Arena?

Other hot AI repos in Research: Scientific Agent Skills, PersonaPlex, PageIndex, Book to Skill, Scientific Figure Library.

Is Loop Arena safe to run?

Deduped does not own, audit or vouch for it. Read the README and license, run it on your own systems first and review every command before you run it.