DEDUPED Hot AI repos, turned into use cases Get the Monday 5

How to set a token budget that finishes reasoning and answers with Halogen Server

Pick max_tokens so the model's thinking does not use up the whole reply budget.

Short answer: Read the per-request default of 8192 and the HALOGEN_MAX_TOKENS_CAP default of 65536 in the configuration table; then send a request with max_completion_tokens, max_output_tokens, or max_tokens set to a larger value; check finish_reason in the reply, and read reasoning_content if content is empty. All steps.

Who it is forA developer whose app gets empty replies from the local model because thinking tokens ran out the budget.

The problemThe model reasons before replying, and a budget that runs out mid-thought returns empty content with finish_reason set to length.

The outcomeRequests finish with a visible answer, and oversized budgets are rejected instead of silently cut.

What you need first

  • AMD Strix Halo machine (Radeon 8060S class GPU, gfx1151) with /dev/kfd and /dev/dri available
  • Podman (or Docker with the video and render group flags described in the README)
  • About 36 GB of free disk for the 35.9 GB checkpoint, plus the 3.5 GB image
  • Python with pip to install the Hugging Face CLI

Steps

  1. Read the per-request default of 8192 and the HALOGEN_MAX_TOKENS_CAP default of 65536 in the configuration table.
  2. Send a request with max_completion_tokens, max_output_tokens, or max_tokens set to a larger value.
  3. Check finish_reason in the reply, and read reasoning_content if content is empty.
  4. Raise the budget in the request body if the reply shows finish_reason of length.
  5. Set HALOGEN_MAX_TOKENS_CAP on the container if your app needs a different ceiling.

Prompt for your agent

In the repo peonist-ai/halogen-server, send the prompt <prompt> to the local endpoint at http://localhost:8731/v1/chat/completions with a max_completion_tokens value of <tokens>. Expected output: a reply with finish_reason of stop and a non-empty content field. Verify that a budget above the cap returns a 400 error rather than a truncated answer.

Caveat. The 8192 default was enough for every ordinary prompt the author measured, so longer reasoning tasks may still need a larger value.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for peonist-ai/halogen-server and "Set a token budget that finishes reasoning and answers"

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer