DEDUPED Hot AI repos, turned into use cases Get the Monday 5

How to run the core exam with a coding agent with Embodied First Exam

Score a coding agent on the 449-task core set of robot tasks in one Harbor command.

Short answer: Clone the repository with its submodules: git clone --recurse-submodules https://github.com/mll-lab-nu/embodied-first-exam.git; then Enter the folder and install Harbor with this repository's flags: uv tool install --force -e . --with-executables-from harbor; copy the example environment file and add your model API key to .env: cp .env.example .env. All steps.

  • Top pick
  • Research
  • Intermediate
  • 1 hour
  • From the README
  • Codex
  • Claude Code

Who it is forA robotics researcher who wants a baseline score for a code agent on physical-world tasks before comparing models.

The problemCode agents are tested on digital tasks, and there is no standard way to see how they perform when writing code that drives a robot in a simulated scene.

The outcomeA scored run of the 449 core tasks, each answered with a program and checked in a separate sandbox.

What you need first

  • Linux x86_64 host with git, uv, and Docker Engine with Compose v2 and BuildKit (or rootless podman with -e podman)
  • NVIDIA driver and NVIDIA Container Toolkit for any GPU task
  • An API key for the agent's model in .env, or a ChatGPT or Claude subscription
  • Enough free disk for the benchmark images (1 to 150 GB depending on the suite)

Steps

  1. Clone the repository with its submodules: git clone --recurse-submodules https://github.com/mll-lab-nu/embodied-first-exam.git
  2. Enter the folder and install Harbor with this repository's flags: uv tool install --force -e . --with-executables-from harbor
  3. Copy the example environment file and add your model API key to .env: cp .env.example .env
  4. Run the core set in its default mode: harbor run -d embodied-first-exam -a codex -m openai/gpt-6-astra

Prompt for your agent

Use the repo mll-lab-nu/embodied-first-exam to run the core set with the agent <agent name> and model <model id>. Report the number of tasks completed and the overall score from the Harbor output. Verify that the run used 449 tasks before you report the result.

Caveat. The first run pulls benchmark images, and some suites need an RTX-class GPU, so check the README table of GPU and disk needs before starting the full set.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for mll-lab-nu/embodied-first-exam and "Run the core exam with a coding agent"

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer