What do you want to do?
3 use cases
-
Reproduce a BEHAVIOR task mean Q-score
Run the five fixed BEHAVIOR instances for one task and compare the mean with the reference value.
- Robo Harness
- Research
- Advanced
View prompt
In the RoboHarness repo, run the task <task id from ./run.sh --list> with --dry-run first, then the full five-instance run into a new run directory, then generate the validation report. Report the task mean from runs/<run>/summary.json and whether --require-match passed. Verify that all five instance IDs are 301, 304, 306, 308 and 310 before you start.
-
Watch one agent rollout in a browser
Run a single BEHAVIOR instance and open the live interface through an SSH tunnel.
- Robo Harness
- Research
- Advanced
View prompt
In the RoboHarness repo, start a single-instance trial of task <task id> with --instances 301 in a new run directory that does not already exist. Report the path of instance_301/ and the last lines of its error log. Verify that the run directory is new before launching.
-
Point the harness at your own model endpoint
Connect the Claude Code harness to a self-hosted Anthropic-compatible model server.
- Robo Harness
- Claude Code
- Codex
- Research
- Intermediate
View prompt
In the RoboHarness repo, set the model URL and model ID from the environment variables for <model server host>, send the small Messages test request from the README, and then run the task01 dry run. Report the model_url, model, gpu and port values from the dry-run output. Verify that the URL has no /v1 suffix before you run the test.
How to use Robo Harness
- Check you have what it needs: Linux x86-64 machine with an NVIDIA RTX GPU and the CUDA 12.4 toolkit (the validation host had 48 GB VRAM per GPU); Python 3.11 and Claude Code 2.1.259 for the paper configuration; Licensed BEHAVIOR scene data, plus the Isaac Sim terms that the README says apply.
- Open BinceQu/RoboHarness on GitHub and read the README and license.
git clone --recurse-submodules https://github.com/BinceQu/RoboHarness.git./scripts/setup.shcp configs/example.json configs/local.json./run.sh --list
These steps come from our top use case, Reproduce a BEHAVIOR task mean Q-score, which also has a copy-paste agent prompt.
Get 5 new use cases every morning. Free, one short email, unsubscribe anytime.
Get them freeAbout this repo
- Who it is forRobotics and embodied-AI researchers who want to reproduce or extend the BEHAVIOR evaluation of LLM agents on household manipulation.
- Why it is hot
- LanguagePython
- Sourcegithub.com/BinceQu/RoboHarness · Back to Hot repos
Requirements
- Linux x86-64 machine with an NVIDIA RTX GPU and the CUDA 12.4 toolkit (the validation host had 48 GB VRAM per GPU)
- Python 3.11 and Claude Code 2.1.259 for the paper configuration
- Licensed BEHAVIOR scene data, plus the Isaac Sim terms that the README says apply
- A separately running model server with an Anthropic-compatible /v1/messages endpoint that supports image input, tool calls and streaming
- Enough local disk for the BEHAVIOR data and the Hugging Face assets
Maintain Robo Harness? Add the Featured on Deduped badge
Paste this into your README:
[](https://dedupedai.com/use-cases/bincequ-roboharness/?ref=badge)
Questions about Robo Harness
What is Robo Harness?
RoboHarness is a Python harness that lets LLM agents control simulated robot manipulation tasks in BEHAVIOR through a visual-geometric control panel, using tracked image keypoints and depth back-projection. It is an open-source project on GitHub: https://github.com/BinceQu/RoboHarness.
Who is Robo Harness for?
Robotics and embodied-AI researchers who want to reproduce or extend the BEHAVIOR evaluation of LLM agents on household manipulation.
What can I do with Robo Harness?
3 use cases so far, for example: Reproduce a BEHAVIOR task mean Q-score; Watch one agent rollout in a browser; Point the harness at your own model endpoint. Each has steps and a copy-paste agent prompt.
What do I need to run Robo Harness?
Linux x86-64 machine with an NVIDIA RTX GPU and the CUDA 12.4 toolkit (the validation host had 48 GB VRAM per GPU); Python 3.11 and Claude Code 2.1.259 for the paper configuration; Licensed BEHAVIOR scene data, plus the Isaac Sim terms that the README says apply; A separately running model server with an Anthropic-compatible /v1/messages endpoint that supports image input, tool calls and streaming.
What are alternatives to Robo Harness?
Other hot AI repos in Research: Scientific Agent Skills, Speaker Mem R1, DSH Council, Evo Bot, AI Engineering Course.
Is Robo Harness safe to run?
Deduped does not own, audit or vouch for it. Read the README and license, run it on your own systems first and review every command before you run it.