DEDUPED Hot AI repos, turned into use cases Get the Monday 5

How to grade a post-trained model with the task's own grader with Aibuildai LLM Posttrain Agent

Score a deliverable model with the frozen grader that the run trained against.

Who it is forA reviewer or ML engineer who needs an independent score for a model produced by an aibuildai run.

The problemA run's reported progress is only useful if the final model is measured with the same grader and on the full test split.

The outcomeA grader result for the merged model on the full GSM8K test split (--limit -1), computed with the run's own Program environment.

What you need first

  • Linux x86_64 with a systemd user session and Python 3.11 to 3.13
  • A C toolchain (gcc, make) with zlib and readline headers, plus bwrap (bubblewrap) with unprivileged user namespaces enabled
  • A Claude Code login or an Anthropic API key, or an Anthropic-compatible endpoint and its key
  • A CUDA GPU is recommended
  • Network access to GitHub releases and ftp.postgresql.org

Steps

  1. Set the run home with H=~/gsm8k-example/playground/gsm8k-qwen3-1.7b-base/
  2. Confirm the deliverable exists at $H/deliverable/final_model
  3. Run $H/program-environment/env/bin/python ~/gsm8k-example/task/grade/evaluate.py --model-path $H/deliverable/final_model --limit -1 (inferred, check the README)
  4. Compare the score with the base model score of 0.105 on the first 200 problems

Prompt for your agent

For the aibuildai-llm-posttrain-agent GSM8K example, run the grader in the run's Program environment against deliverable/final_model with --limit -1. Expected output: a grader score for the full test split. Verify that the model path exists and that the score is reported on the full split before stating the result.

Caveat. The README only gives the base model's score on the first 200 problems, so compare the trained model on the same split or note the difference in split size.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for aibuildai-inc/aibuildai-llm-posttrain-agent and "Grade a post-trained model with the task's own grader"

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer