DEDUPED Hot AI repos, turned into use cases Get 5 a day

How to find why long agent runs stall, truncate, or time out with Crux

Use the documented failure modes and trial outcome classes to explain a failed terminal run.

Who it is forA developer whose long-running terminal agent runs keep stopping partway, with truncated output, stalls, or timeouts.

The problemLong runs fail in several ways at once, and it is hard to tell whether the cause is the context, the output limit, or the runtime.

The outcomeA short diagnosis of each failed run that maps it to context compaction, truncated output, or runtime idle time, with the matching fix from the agent layer.

What you need first

  • uv installed, since the benchmark is set up with `uv sync`
  • harbor installed, since the benchmark runs through `harbor run`
  • Access to the terminal-bench/terminal-bench-2-1 dataset
  • A model endpoint you can reach through the `openai/` model prefix used in the README command

Steps

  1. Read the 'What changed from pi' section of the README to list the four areas: context engine, agent loop, runtime, and tools.
  2. Run the failed task through the benchmark with harbor run --dataset terminal-bench/terminal-bench-2-1 --agent crux.pi_agent:CruxPiAgent --model openai/<your-model>. (inferred, check the README)
  3. Check the harness trial outcome for each run and sort failures into compaction failures, truncated outputs, or idle time in timed-out runs.
  4. Compare the failing pattern with the matching fix in the agent layer, such as the stream idle watchdog or the two-phase recovery from truncated output. (inferred, check the README)

Prompt for your agent

Use the alanhuangyoo/crux repository. Read the 'What changed from pi' section of its README, then run the failing <task> from Terminal-Bench 2.1 through harbor with crux.pi_agent:CruxPiAgent. Report which failure class the trial outcome falls into (compaction, truncated output, or idle time) and name the matching agent-layer feature, then verify the classification against the trial output.

Caveat. The README reports improvements in these failure rates as measured by the author, and it does not describe how to tune each feature for a different model or setup.

Share: X LinkedIn Hacker News Email Copy link

Disclaimer

Deduped lists open-source repositories that belong to other people. We do not own, maintain, audit, or vouch for any of them, and a repo appearing here is not an endorsement or a safety check. The use cases, steps, and prompts are written by AI and may be incomplete, outdated, or just wrong. Where a use case is marked "Seen in the wild", we link to public posts where someone described something similar, but we have not verified those posts and the use case itself is still our AI's own write-up. Use all of this at your own risk. Only run code, commands, or agents on systems and accounts you are authorized to use, and read the code and every command before you run it, especially anything that touches credentials, files, or the network. Nothing here is professional, security, legal, or financial advice. All trademarks and project names belong to their respective owners, and their use here does not imply affiliation. Content is provided as is, without warranties of any kind. Found a mistake, a safety problem, or something you want removed? Email [email protected] and we will look at it promptly. Full disclaimer page