DEDUPED Hot AI repos, turned into use cases Get the Monday 5

How to test one benchmark suite with a single command with Embodied First Exam

Run only one suite, such as RoboARC, to check a setup before the full exam.

Short answer: Install and configure as in the core exam: clone the repository, run uv tool install --force -e . --with-executables-from harbor, and set your key in .env; then check the GPU rules for your suite in the README table, for RoboARC an NVIDIA driver 580 or newer with one GPU per container; run one suite, here RoboARC with 196 task directories: harbor run -d embodied-first-exam@all -p tasks/roboarc -a codex -m openai/gpt-6-astra. All steps.

  • Research
  • Intermediate
  • 1 hour 30 minutes
  • From the README
  • Codex
  • Claude Code

Who it is forAn engineer who has a GPU workstation and wants to confirm the setup works on one suite before spending time on all 2224 task directories.

The problemThe full exam is large, and a setup problem found late wastes hours of image pulls and runs.

The outcomeA short, verified run on one suite that shows whether the host, Docker, GPU, and model key are working.

What you need first

  • Linux x86_64 host with git, uv, and Docker Engine with Compose v2 and BuildKit (or rootless podman with -e podman)
  • NVIDIA driver and NVIDIA Container Toolkit for any GPU task
  • An API key for the agent's model in .env, or a ChatGPT or Claude subscription
  • Enough free disk for the benchmark images (1 to 150 GB depending on the suite)

Steps

  1. Install and configure as in the core exam: clone the repository, run uv tool install --force -e . --with-executables-from harbor, and set your key in .env
  2. Check the GPU rules for your suite in the README table, for RoboARC an NVIDIA driver 580 or newer with one GPU per container
  3. Run one suite, here RoboARC with 196 task directories: harbor run -d embodied-first-exam@all -p tasks/roboarc -a codex -m openai/gpt-6-astra

Prompt for your agent

In the repo mll-lab-nu/embodied-first-exam, run only the suite at tasks/<suite folder> with the agent codex and the model <model id>. Report the pass count and any setup error from the Harbor log. Verify that the task count matches the number of task directories in that suite.

Caveat. RoboARC needs a 24 GB GPU for knot_untying and rope_threading, so a smaller card may fail on those tasks even when the rest of the suite runs.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for mll-lab-nu/embodied-first-exam and "Test one benchmark suite with a single command"

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer