DEDUPED Hot AI repos, turned into use cases Get the Monday 5

How to measure prefill and decode speed on your own machine with Halogen Server

Run the bundled sweep and bench tools to get reproducible tokens-per-second figures.

Short answer: Download the checkpoint and tokenizer as described for the endpoint setup; then run the ten-shape benchmark with podman run --rm --device /dev/kfd --device /dev/dri --group-add keep-groups --security-opt seccomp=unconfined --ipc=host -v /path/to/models:/models:ro -v /path/to/tokenizer:/tokenizer:ro ghcr.io/peonist-ai/halogen:0.1.4 bench dflash2 256 low 3. All steps.

Who it is forA performance-minded engineer who needs real throughput numbers for their own Strix Halo box before choosing a local model setup.

The problemPublished speed figures for local models vary by prompt and configuration, so quoted numbers are hard to compare.

The outcomeA set of prefill and decode figures measured on the user's own hardware, with the prompt set and version named.

What you need first

  • AMD Strix Halo machine (Radeon 8060S class GPU, gfx1151) with /dev/kfd and /dev/dri available
  • Podman (or Docker with the video and render group flags described in the README)
  • About 36 GB of free disk for the 35.9 GB checkpoint, plus the 3.5 GB image
  • Python with pip to install the Hugging Face CLI

Steps

  1. Download the checkpoint and tokenizer as described for the endpoint setup.
  2. Run the ten-shape benchmark with podman run --rm --device /dev/kfd --device /dev/dri --group-add keep-groups --security-opt seccomp=unconfined --ipc=host -v /path/to/models:/models:ro -v /path/to/tokenizer:/tokenizer:ro ghcr.io/peonist-ai/halogen:0.1.4 bench dflash2 256 low 3. (inferred, check the README)
  3. Run the llama-bench-shaped sweep with podman run --rm ... ghcr.io/peonist-ai/halogen:0.1.4 sweep -p 512,2048,8192 -n 128,256 -d dflash2,mtp -r 3 (add the same device and volume flags as the bench command). (inferred, check the README)
  4. Add --json to the sweep command if you want machine-readable output.
  5. Record the version, the drafter, and the prompt set next to every number you publish.

Prompt for your agent

Using the repo peonist-ai/halogen-server, run the bench and sweep commands from its README against the models at <models_path> on this machine. Expected output: prefill tokens per second for each prompt length and decode mean and range for each drafter. Verify that each reported figure names the version 0.1.4 and the prompt set.

Caveat. Decode speed on this engine varies with the prompt type, so a single number is not meaningful unless the prompt set is named.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for peonist-ai/halogen-server and "Measure prefill and decode speed on your own machine"

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer