# AI use cases for Crux (alanhuangyoo/crux) Repo: [alanhuangyoo/crux](https://github.com/alanhuangyoo/crux) | Use cases: 3 | Page: https://dedupedai.com/use-cases/alanhuangyoo-crux/ Crux is a coding agent built on the pi agent, re-engineered for long-horizon terminal tasks, with a Terminal-Bench 2.1 benchmark harness. This file holds every use case of this repo. Each one starts with a "## " heading and ends with its URL. - Benchmark a self-hosted model on Terminal-Bench 2.1: https://dedupedai.com/use-cases/alanhuangyoo-crux/benchmark-self-hosted-model-on-terminal-bench/ - Test whether an agent change helps, task by task: https://dedupedai.com/use-cases/alanhuangyoo-crux/compare-agent-versions-task-by-task/ - Find why long agent runs stall, truncate, or time out: https://dedupedai.com/use-cases/alanhuangyoo-crux/diagnose-stalled-or-truncated-agent-runs/ ## How to benchmark a self-hosted model on Terminal-Bench 2.1 with Crux Repo: [alanhuangyoo/crux](https://github.com/alanhuangyoo/crux) | Difficulty: Advanced | Time: 120 minutes | Basis: From the README Run the Crux agent on 89 Terminal-Bench 2.1 tasks to get a pass@1 score for your own model. **Who it is for:** An ML engineer who serves an open-weight coding model locally and needs a repeatable score before deciding whether to use it for agent work. **The problem:** Agent scores vary with the harness, so a model's raw ability is hard to compare without the same tasks and the same agent setup. **The outcome:** A pass@1 score for your model running inside the Crux agent on the Terminal-Bench 2.1 task set, comparable to the README results. ### Steps 1. Clone the repository and open the benchmark folder with `cd benchmark`. 2. Install the harness dependencies with `uv sync`. 3. Finish the `--model` argument after `openai/` with the name of your served model, since the README command is cut off there. (inferred, check the README) 4. Run `harbor run --dataset terminal-bench/terminal-bench-2-1 --agent crux.pi_agent:CruxPiAgent --model openai/`. (inferred, check the README) ### Prompt to paste into your agent ```text Use the alanhuangyoo/crux repository. Go to its benchmark folder, run `uv sync`, then run harbor against the terminal-bench/terminal-bench-2-1 dataset with agent crux.pi_agent:CruxPiAgent and model openai/. Report the pass@1 score over the full run and the number of tasks evaluated, and confirm the run covered 89 tasks. ``` ### Caveat The README gives the model argument only as far as openai/, so you must supply the rest yourself and the full run covers 89 tasks, which takes a long time. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/alanhuangyoo-crux/benchmark-self-hosted-model-on-terminal-bench/ ## How to test whether an agent change helps, task by task with Crux Repo: [alanhuangyoo/crux](https://github.com/alanhuangyoo/crux) | Difficulty: Advanced | Time: 240 minutes | Basis: From the README Compare two agent versions on the same tasks with a paired sign test instead of one total score. **Who it is for:** An engineer changing the agent layer of a pi-based coding agent who needs to know whether an edit is a real improvement or noise. **The problem:** A single pass rate can move by chance, so an edit can look like a gain or a loss without being one. **The outcome:** A task-by-task comparison of two agent versions with a paired sign test that shows which changes are meaningful. ### Steps 1. Clone the repository and open the benchmark folder with `cd benchmark`. 2. Install the harness dependencies with `uv sync`. 3. Run the baseline version and the changed version with `harbor run --dataset terminal-bench/terminal-bench-2-1 --agent crux.pi_agent:CruxPiAgent --model openai/`, keeping the model and dataset the same. (inferred, check the README) 4. Use the harness comparison step in the benchmark folder to compare the two runs task by task with the paired sign test. (inferred, check the README) ### Prompt to paste into your agent ```text Use the alanhuangyoo/crux repository benchmark folder. Run the same Terminal-Bench 2.1 dataset and model for two agent versions, and , then compare them task by task with the paired sign test. Output the per-task outcome counts and the test result, and verify that both runs used the same model and task set. ``` ### Caveat The README describes the paired sign test but does not show the exact command for it, so check the benchmark folder for how to invoke the comparison. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/alanhuangyoo-crux/compare-agent-versions-task-by-task/ ## How to find why long agent runs stall, truncate, or time out with Crux Repo: [alanhuangyoo/crux](https://github.com/alanhuangyoo/crux) | Difficulty: Intermediate | Time: 60 minutes | Basis: From the README Use the documented failure modes and trial outcome classes to explain a failed terminal run. **Who it is for:** A developer whose long-running terminal agent runs keep stopping partway, with truncated output, stalls, or timeouts. **The problem:** Long runs fail in several ways at once, and it is hard to tell whether the cause is the context, the output limit, or the runtime. **The outcome:** A short diagnosis of each failed run that maps it to context compaction, truncated output, or runtime idle time, with the matching fix from the agent layer. ### Steps 1. Read the 'What changed from pi' section of the README to list the four areas: context engine, agent loop, runtime, and tools. 2. Run the failed task through the benchmark with `harbor run --dataset terminal-bench/terminal-bench-2-1 --agent crux.pi_agent:CruxPiAgent --model openai/`. (inferred, check the README) 3. Check the harness trial outcome for each run and sort failures into compaction failures, truncated outputs, or idle time in timed-out runs. 4. Compare the failing pattern with the matching fix in the agent layer, such as the stream idle watchdog or the two-phase recovery from truncated output. (inferred, check the README) ### Prompt to paste into your agent ```text Use the alanhuangyoo/crux repository. Read the 'What changed from pi' section of its README, then run the failing from Terminal-Bench 2.1 through harbor with crux.pi_agent:CruxPiAgent. Report which failure class the trial outcome falls into (compaction, truncated output, or idle time) and name the matching agent-layer feature, then verify the classification against the trial output. ``` ### Caveat The README reports improvements in these failure rates as measured by the author, and it does not describe how to tune each feature for a different model or setup. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/alanhuangyoo-crux/diagnose-stalled-or-truncated-agent-runs/