DEDUPED Hot AI repos, turned into use cases Get the Monday 5
137 stars · Python

How to use Halogen Server: what you can do with it

What is Halogen Server? Halogen is a Python-based engine that runs the Qwen3.8-27B model on AMD Strix Halo (gfx1151) GPUs and serves an OpenAI-compatible API on port 8731.

How do I use it? Install the Hugging Face CLI with pip install -U "huggingface_hub[cli]"; then download the weights and tokenizer with hf download peonist-ai/halogen-qwen3.8-27b --local-dir ~/halogen-models. Full steps below.

Trended earlier

View repo on GitHub

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer

What do you want to do?

4 use cases

  • Run a local OpenAI-compatible Qwen3.8-27B server

    Start the halogen container on Strix Halo and call a local chat endpoint on port 8731.

    View prompt
    In the repo peonist-ai/halogen-server, start the ghcr.io/peonist-ai/halogen:0.1.4 container with the checkpoint directory at <models_path> and the tokenizer at <models_path>/tokenizer, then send the prompt <prompt> to http://localhost:8731/v1/chat/completions. Expected output: a chat completion JSON with a non-empty message content. Verify that finish_reason is not "length".

  • Measure prefill and decode speed on your own machine

    Run the bundled sweep and bench tools to get reproducible tokens-per-second figures.

    View prompt
    Using the repo peonist-ai/halogen-server, run the bench and sweep commands from its README against the models at <models_path> on this machine. Expected output: prefill tokens per second for each prompt length and decode mean and range for each drafter. Verify that each reported figure names the version 0.1.4 and the prompt set.

  • Continue long chats without re-processing the context

    Use the prompt cache so follow-up turns on a long document resume instead of re-prefilling.

    View prompt
    In the repo peonist-ai/halogen-server, run the halogen:0.1.4 server, send a long document from <document_path> as the first chat turn, then send the follow-up question <question> in the same conversation. Expected output: a faster time to first token on the follow-up than on the first turn. Verify that the follow-up answer is the same as a cold run with the cache disabled.

  • Set a token budget that finishes reasoning and answers

    Pick max_tokens so the model's thinking does not use up the whole reply budget.

    View prompt
    In the repo peonist-ai/halogen-server, send the prompt <prompt> to the local endpoint at http://localhost:8731/v1/chat/completions with a max_completion_tokens value of <tokens>. Expected output: a reply with finish_reason of stop and a non-empty content field. Verify that a budget above the cap returns a 400 error rather than a truncated answer.

Quick start with Halogen Server

  1. Check you have what it needs: AMD Strix Halo machine (Radeon 8060S class GPU, gfx1151) with /dev/kfd and /dev/dri available; Podman (or Docker with the video and render group flags described in the README); About 36 GB of free disk for the 35.9 GB checkpoint, plus the 3.5 GB image.
  2. Open peonist-ai/halogen-server on GitHub and read the README and license.
  3. Install the Hugging Face CLI with pip install -U "huggingface_hub[cli]".
  4. Download the weights and tokenizer with hf download peonist-ai/halogen-qwen3.8-27b --local-dir ~/halogen-models. (inferred, check the README)
  5. Start the server with podman run --rm -p 8731:8731 --device /dev/kfd --device /dev/dri --group-add keep-groups --security-opt seccomp=unconfined --ipc=host -v ~/halogen-models:/models:ro -v ~/halogen-models/tokenizer:/tokenizer:ro ghcr.io/peonist-ai/halogen:0.1.4. (inferred, check the README)
  6. Send a chat request with curl to http://localhost:8731/v1/chat/completions using a JSON body with a messages list. (inferred, check the README)

These steps come from our top use case, Run a local OpenAI-compatible Qwen3.8-27B server, which also has a copy-paste agent prompt.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for peonist-ai/halogen-server

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Halogen Server alternatives · Best AI repos for Software and DevOps

About this repo

Requirements

  • AMD Strix Halo machine (Radeon 8060S class GPU, gfx1151) with /dev/kfd and /dev/dri available
  • Podman (or Docker with the video and render group flags described in the README)
  • About 36 GB of free disk for the 35.9 GB checkpoint, plus the 3.5 GB image
  • Python with pip to install the Hugging Face CLI

Add this badge to your README

Maintain Halogen Server? Show readers where to find these use cases. Use cases on Deduped

Markdown 

[![Use cases on Deduped](https://dedupedai.com/badge/peonist-ai/halogen-server.svg)](https://dedupedai.com/use-cases/peonist-ai-halogen-server/?utm_source=github&utm_medium=badge&utm_campaign=peonist-ai-halogen-server)

HTML 

<a href="https://dedupedai.com/use-cases/peonist-ai-halogen-server/?utm_source=github&amp;utm_medium=badge&amp;utm_campaign=peonist-ai-halogen-server"><img src="https://dedupedai.com/badge/peonist-ai/halogen-server.svg" alt="Use cases on Deduped" height="20"></a>

A static image, no tracking script. About the badge

Questions about Halogen Server

What is Halogen Server?

Halogen is a Python-based engine that runs the Qwen3.8-27B model on AMD Strix Halo (gfx1151) GPUs and serves an OpenAI-compatible API on port 8731. It is an open-source project on GitHub: https://github.com/peonist-ai/halogen-server.

How do I use Halogen Server?

Install the Hugging Face CLI with pip install -U "huggingface_hub[cli]"; then download the weights and tokenizer with hf download peonist-ai/halogen-qwen3.8-27b --local-dir ~/halogen-models. The use cases on this page give the full steps and a copy-paste agent prompt for each job.

Who is Halogen Server for?

Developers and AI-tool builders who own AMD Strix Halo hardware and want a fast local Qwen3.8-27B endpoint.

What can I do with Halogen Server?

4 use cases so far, for example: Run a local OpenAI-compatible Qwen3.8-27B server; Measure prefill and decode speed on your own machine; Continue long chats without re-processing the context. Each has steps and a copy-paste agent prompt.

What do I need to run Halogen Server?

AMD Strix Halo machine (Radeon 8060S class GPU, gfx1151) with /dev/kfd and /dev/dri available; Podman (or Docker with the video and render group flags described in the README); About 36 GB of free disk for the 35.9 GB checkpoint, plus the 3.5 GB image; Python with pip to install the Hugging Face CLI.

What are alternatives to Halogen Server?

Other hot AI repos in Software and DevOps: Deep Infer, Harness2, Orca, Opencode, Graphify.

Is Halogen Server safe to run?

Deduped does not own, audit or vouch for it. Read the README and license, run it on your own systems first and review every command before you run it.