How to use Embodied First Exam: what you can do with it

4 use cases

• Run the core exam with a coding agent

  Score a coding agent on the 449-task core set of robot tasks in one Harbor command.

  • REPO mll-lab-nu/embodied-first-exam
  • Codex
  • Claude Code
  • Research
  • Intermediate
  View prompt

  Use the repo mll-lab-nu/embodied-first-exam to run the core set with the agent <agent name> and model <model id>. Report the number of tasks completed and the overall score from the Harbor output. Verify that the run used 449 tasks before you report the result.

  Share Email me this

• Test one benchmark suite with a single command

  Run only one suite, such as RoboARC, to check a setup before the full exam.

  • REPO mll-lab-nu/embodied-first-exam
  • Codex
  • Claude Code
  • Research
  • Intermediate
  View prompt

  In the repo mll-lab-nu/embodied-first-exam, run only the suite at tasks/<suite folder> with the agent codex and the model <model id>. Report the pass count and any setup error from the Harbor log. Verify that the task count matches the number of task directories in that suite.

  Share Email me this

• Run the exam on a ChatGPT or Claude subscription

  Use a subscription login in place of an API key to run the exam with codex or claude-code.

  • REPO mll-lab-nu/embodied-first-exam
  • Codex
  • Claude Code
  • Research
  • Intermediate
  View prompt

  In the repo mll-lab-nu/embodied-first-exam, run the core set with the subscription login for <agent name> using the matching command from the README. Report the task count and score. Verify that the login token is stored only in your local .env or shell and is not printed in the output.

  Share Email me this

• Test an agent on the Embodied Safety Exam without a GPU

  Run the Embodied Safety Exam, which the README lists as needing no GPU, to test agent behavior on robot tasks.

  • REPO mll-lab-nu/embodied-first-exam
  • Codex
  • Claude Code
  • Research
  • Intermediate
  View prompt

  In the repo mll-lab-nu/embodied-first-exam, run the Embodied Safety Exam suite with the agent codex and the model <model id> on a host without a GPU. Report which tasks passed and which failed, and any safety-related failures. Verify that no GPU was requested in the run output.

  Share Email me this

Quick start with Embodied First Exam

1. Check you have what it needs: Linux x86_64 host with git, uv, and Docker Engine with Compose v2 and BuildKit (or rootless podman with -e podman); NVIDIA driver and NVIDIA Container Toolkit for any GPU task; An API key for the agent's model in .env, or a ChatGPT or Claude subscription.
2. Open mll-lab-nu/embodied-first-exam on GitHub and read the README and license.
3. Clone the repository with its submodules: git clone --recurse-submodules https://github.com/mll-lab-nu/embodied-first-exam.git
4. Enter the folder and install Harbor with this repository's flags: uv tool install --force -e . --with-executables-from harbor
5. Copy the example environment file and add your model API key to .env: cp .env.example .env
6. Run the core set in its default mode: harbor run -d embodied-first-exam -a codex -m openai/gpt-6-astra

These steps come from our top use case, Run the core exam with a coding agent, which also has a copy-paste agent prompt.

Using the Deduped MCP, get the setup steps for mll-lab-nu/embodied-first-exam

claude mcp add --transport http deduped https://dedupedai.com/mcp

Embodied First Exam alternatives · Best AI repos for Research

About this repo

• Who it is forRobotics and AI researchers, and teams building coding agents, who need to measure how well an agent writes code that controls robots.
• Why it is hot178 stars, launched 38 days ago
• LanguagePython
• Sourcegithub.com/mll-lab-nu/embodied-first-exam · Back to Hot repos

Requirements

• Linux x86_64 host with git, uv, and Docker Engine with Compose v2 and BuildKit (or rootless podman with -e podman)
• NVIDIA driver and NVIDIA Container Toolkit for any GPU task
• An API key for the agent's model in .env, or a ChatGPT or Claude subscription
• Enough free disk for the benchmark images (1 to 150 GB depending on the suite)

Related repos

• #8 this week REPO K-Dense-AI/scientific-agent-skills Scientific Agent Skills A library of 177 Agent Skills that give AI agents documented, validated guidance for scientific packages, databases, and research workflows.
• Trended earlier REPO NVIDIA/personaplex PersonaPlex PersonaPlex is NVIDIA's real-time, full-duplex speech-to-speech conversational model based on Moshi that controls persona through text role prompts and audio voice conditioning.
• Trended earlier REPO VectifyAI/PageIndex PageIndex PageIndex is a vectorless, reasoning-based RAG system that builds a tree index of long documents and lets an LLM search it like a human reader, with local and cloud SDK modes.