# AI use cases for Embodied First Exam (mll-lab-nu/embodied-first-exam) Repo: [mll-lab-nu/embodied-first-exam](https://github.com/mll-lab-nu/embodied-first-exam) | Use cases: 4 | Page: https://dedupedai.com/use-cases/mll-lab-nu-embodied-first-exam/ A benchmark called Embodied First Exam that tests code agents on frozen robot tasks, where each answer is a program run and verified in a separate sandbox, built to run on Harbor. This file holds every use case of this repo. Each one starts with a "## " heading and ends with its URL. - Run the core exam with a coding agent: https://dedupedai.com/use-cases/mll-lab-nu-embodied-first-exam/run-core-exam-with-coding-agent/ - Test one benchmark suite with a single command: https://dedupedai.com/use-cases/mll-lab-nu-embodied-first-exam/test-one-benchmark-suite/ - Run the exam on a ChatGPT or Claude subscription: https://dedupedai.com/use-cases/mll-lab-nu-embodied-first-exam/run-exam-on-subscription-login/ - Test an agent on the Embodied Safety Exam without a GPU: https://dedupedai.com/use-cases/mll-lab-nu-embodied-first-exam/test-safety-suite-without-gpu/ ## How to run the core exam with a coding agent with Embodied First Exam Repo: [mll-lab-nu/embodied-first-exam](https://github.com/mll-lab-nu/embodied-first-exam) | Difficulty: Intermediate | Time: 1 hour | Basis: From the README Score a coding agent on the 449-task core set of robot tasks in one Harbor command. **Who it is for:** A robotics researcher who wants a baseline score for a code agent on physical-world tasks before comparing models. **The problem:** Code agents are tested on digital tasks, and there is no standard way to see how they perform when writing code that drives a robot in a simulated scene. **The outcome:** A scored run of the 449 core tasks, each answered with a program and checked in a separate sandbox. ### Steps 1. Clone the repository with its submodules: `git clone --recurse-submodules https://github.com/mll-lab-nu/embodied-first-exam.git` 2. Enter the folder and install Harbor with this repository's flags: `uv tool install --force -e . --with-executables-from harbor` 3. Copy the example environment file and add your model API key to `.env`: `cp .env.example .env` 4. Run the core set in its default mode: `harbor run -d embodied-first-exam -a codex -m openai/gpt-6-astra` ### Prompt to paste into your agent ```text Use the repo mll-lab-nu/embodied-first-exam to run the core set with the agent and model . Report the number of tasks completed and the overall score from the Harbor output. Verify that the run used 449 tasks before you report the result. ``` ### Caveat The first run pulls benchmark images, and some suites need an RTX-class GPU, so check the README table of GPU and disk needs before starting the full set. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/mll-lab-nu-embodied-first-exam/run-core-exam-with-coding-agent/ ## How to test one benchmark suite with a single command with Embodied First Exam Repo: [mll-lab-nu/embodied-first-exam](https://github.com/mll-lab-nu/embodied-first-exam) | Difficulty: Intermediate | Time: 1 hour 30 minutes | Basis: From the README Run only one suite, such as RoboARC, to check a setup before the full exam. **Who it is for:** An engineer who has a GPU workstation and wants to confirm the setup works on one suite before spending time on all 2224 task directories. **The problem:** The full exam is large, and a setup problem found late wastes hours of image pulls and runs. **The outcome:** A short, verified run on one suite that shows whether the host, Docker, GPU, and model key are working. ### Steps 1. Install and configure as in the core exam: clone the repository, run `uv tool install --force -e . --with-executables-from harbor`, and set your key in `.env` 2. Check the GPU rules for your suite in the README table, for RoboARC an NVIDIA driver 580 or newer with one GPU per container 3. Run one suite, here RoboARC with 196 task directories: `harbor run -d embodied-first-exam@all -p tasks/roboarc -a codex -m openai/gpt-6-astra` ### Prompt to paste into your agent ```text In the repo mll-lab-nu/embodied-first-exam, run only the suite at tasks/ with the agent codex and the model . Report the pass count and any setup error from the Harbor log. Verify that the task count matches the number of task directories in that suite. ``` ### Caveat RoboARC needs a 24 GB GPU for knot_untying and rope_threading, so a smaller card may fail on those tasks even when the rest of the suite runs. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/mll-lab-nu-embodied-first-exam/test-one-benchmark-suite/ ## How to run the exam on a ChatGPT or Claude subscription with Embodied First Exam Repo: [mll-lab-nu/embodied-first-exam](https://github.com/mll-lab-nu/embodied-first-exam) | Difficulty: Intermediate | Time: 1 hour | Basis: From the README Use a subscription login in place of an API key to run the exam with codex or claude-code. **Who it is for:** A researcher who already pays for a ChatGPT or Claude subscription and does not want to set up separate API billing for a test run. **The problem:** API keys need a separate billing setup, and some people prefer to run the exam with the login they already use. **The outcome:** An exam run that authenticates with the subscription login and reports results like a key-based run. ### Steps 1. Install Harbor as in the core exam: `uv tool install --force -e . --with-executables-from harbor` 2. For ChatGPT, log in once with `codex login`, then run: `CODEX_FORCE_AUTH_JSON=1 harbor run -d embodied-first-exam -a codex -m openai/gpt-6-astra` 3. For Claude, run `claude setup-token`, put the token it prints in `.env` as CLAUDE_CODE_OAUTH_TOKEN, then run: `CLAUDE_FORCE_OAUTH=1 harbor run -d embodied-first-exam -a claude-code -m anthropic/claude-opus-5-5` (inferred, check the README) ### Prompt to paste into your agent ```text In the repo mll-lab-nu/embodied-first-exam, run the core set with the subscription login for using the matching command from the README. Report the task count and score. Verify that the login token is stored only in your local .env or shell and is not printed in the output. ``` ### Caveat The README lists this as a separate path from the API key setup, so check that your subscription covers the model named in the command before you start. Risk, credentials: The login token is tied to one person's account, so keep it in your own .env and do not share it or run the exam through a pooled account. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/mll-lab-nu-embodied-first-exam/run-exam-on-subscription-login/ ## How to test an agent on the Embodied Safety Exam without a GPU with Embodied First Exam Repo: [mll-lab-nu/embodied-first-exam](https://github.com/mll-lab-nu/embodied-first-exam) | Difficulty: Intermediate | Time: 1 hour | Basis: From the README Run the Embodied Safety Exam, which the README lists as needing no GPU, to test agent behavior on robot tasks. **Who it is for:** A safety-minded engineer on a machine with no GPU who wants to see how a code agent handles safety-focused robot tasks. **The problem:** Many suites need a GPU, which blocks people who only have a CPU machine from testing agent safety on robot tasks. **The outcome:** A run of the safety suite on a CPU-only host, with per-task results from the Harbor output. ### Steps 1. Install Harbor as in the core exam: `uv tool install --force -e . --with-executables-from harbor` 2. Confirm your host has Docker Engine with Compose v2 and BuildKit, since the README lists no GPU needed for Embodied Safety Exam 3. Run the safety suite by pointing -p at its folder under tasks/: `harbor run -d embodied-first-exam@all -p tasks/ -a codex -m openai/gpt-6-astra` (inferred, check the README) ### Prompt to paste into your agent ```text In the repo mll-lab-nu/embodied-first-exam, run the Embodied Safety Exam suite with the agent codex and the model on a host without a GPU. Report which tasks passed and which failed, and any safety-related failures. Verify that no GPU was requested in the run output. ``` ### Caveat The exact folder name for the safety suite is not given in the README excerpt, so confirm it under tasks/ before running the inferred command. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/mll-lab-nu-embodied-first-exam/test-safety-suite-without-gpu/