DEDUPED Hot AI repos, turned into use cases Get the Monday 5
934 stars · C++

How to use Gufo: what you can do with it

What is Gufo? Gufo is an open source local inference engine in C++ built for AMD Strix Halo systems, serving text, audio, image, and video models through an OpenAI-compatible API.

How do I use it? Run hf download unsloth/Qwen3.8-27B-GGUF Qwen3.8-27B-UD-Q8_K_XL.gguf --revision 4ca720788d1e01f1bff70c033e0d0028fd02e502 --repo-type model --local-dir models/Qwen3.8-27B-GGUF to get the main model file. Full steps below.

Trended earlier

View repo on GitHub

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer

What do you want to do?

4 use cases

  • Run Qwen3.8 27B as a private local API on Strix Halo

    Download the model and start a local OpenAI-compatible chat endpoint on port 8080 in one container.

    View prompt
    Using the gufo repo, help me serve the Qwen3.8 27B model on my Strix Halo machine at <host path to models folder>. Follow the Quickstart in the README, then send one chat request to http://localhost:8080/v1/chat/completions. Report the HTTP status and the reply text, and confirm the container logs show no ROCm device error.

  • Point Codex at a local Qwen model through Gufo

    Set the Codex or OpenAI SDK base URL to your local Gufo server so coding sessions stay on your machine.

    View prompt
    In the gufo repo, configure Codex to use the OpenAI-compatible base URL http://localhost:8080/v1 with model <model name>. Send one prompt asking for a one-line Python hello world and report the reply. Verify that the response is streamed and that Gufo logs show a completed request.

  • Build Gufo from source with Nix and run diagnostics

    Compile the production package with Nix, check the Strix Halo setup, and start a local model.

    View prompt
    Clone the gufo repo, run nix build, then run ./result/bin/gufo diagnose and report its full output. Start ./result/bin/gufo serve llm --model <path to GGUF file> and confirm the server reports it is listening.

  • Compare Gufo speed with a llama.cpp baseline

    Run Gufo's benchmark guide against optional llama.cpp reference builds on your own Strix Halo box.

    View prompt
    In the gufo repo, follow docs/BENCHMARKS.md to run the Qwen3.8 27B benchmark on <model path> against the llama.cpp reference. Report prompt processing and token generation rates in tok/s and compare them with the README figures. Verify the same model file and settings were used for both runs.

Quick start with Gufo

  1. Check you have what it needs: Linux x86-64 machine with AMD Strix Halo (gfx1151) and enough unified memory for the chosen model; Podman with rootless GPU device access to /dev/kfd and /dev/dri; Hugging Face CLI (hf) to download model weights.
  2. Open gufo-org/gufo on GitHub and read the README and license.
  3. Run hf download unsloth/Qwen3.8-27B-GGUF Qwen3.8-27B-UD-Q8_K_XL.gguf --revision 4ca720788d1e01f1bff70c033e0d0028fd02e502 --repo-type model --local-dir models/Qwen3.8-27B-GGUF to get the main model file. (inferred, check the README)
  4. Run hf download z-lab/Qwen3.8-27B-DFlash2-GGUF Qwen3.8-27B-DFlash2-Q4_K_M.gguf --revision 2d9571f8ce46e151f61c6499c99dee6079e1d610 --repo-type model --local-dir models/Qwen3.8-27B-DFlash2-GGUF to get the DFlash2 draft model. (inferred, check the README)
  5. Run podman pull ghcr.io/gufo-org/toolboxes/gufo-runtime:latest to fetch the Gufo runtime image.
  6. Run the podman run command from the README Quickstart with --device /dev/kfd, --device /dev/dri, -p 8080:8080, and gufo serve --host 0.0.0.0 --port 8080 llm using the two model paths and --speculative dflash2.

These steps come from our top use case, Run Qwen3.8 27B as a private local API on Strix Halo, which also has a copy-paste agent prompt.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for gufo-org/gufo

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Gufo alternatives · Best AI repos for Software and DevOps

About this repo

  • Who it is forDevelopers and power users with an AMD Ryzen AI MAX+ 395 (Strix Halo) machine who want to run large open models locally on their own hardware.
  • Why it is hot934 stars, launched 76 days ago
  • LanguageC++
  • Sourcegithub.com/gufo-org/gufo · Back to Hot repos

Requirements

  • Linux x86-64 machine with AMD Strix Halo (gfx1151) and enough unified memory for the chosen model
  • Podman with rootless GPU device access to /dev/kfd and /dev/dri
  • Hugging Face CLI (hf) to download model weights
  • Several tens of GB of free disk space for model files

Add this badge to your README

Maintain Gufo? Show readers where to find these use cases. Use cases on Deduped

Markdown 

[![Use cases on Deduped](https://dedupedai.com/badge/gufo-org/gufo.svg)](https://dedupedai.com/use-cases/gufo-org-gufo/?utm_source=github&utm_medium=badge&utm_campaign=gufo-org-gufo)

HTML 

<a href="https://dedupedai.com/use-cases/gufo-org-gufo/?utm_source=github&amp;utm_medium=badge&amp;utm_campaign=gufo-org-gufo"><img src="https://dedupedai.com/badge/gufo-org/gufo.svg" alt="Use cases on Deduped" height="20"></a>

A static image, no tracking script. About the badge

Questions about Gufo

What is Gufo?

Gufo is an open source local inference engine in C++ built for AMD Strix Halo systems, serving text, audio, image, and video models through an OpenAI-compatible API. It is an open-source project on GitHub: https://github.com/gufo-org/gufo.

How do I use Gufo?

Run hf download unsloth/Qwen3.8-27B-GGUF Qwen3.8-27B-UD-Q8_K_XL.gguf --revision 4ca720788d1e01f1bff70c033e0d0028fd02e502 --repo-type model --local-dir models/Qwen3.8-27B-GGUF to get the main model file. The use cases on this page give the full steps and a copy-paste agent prompt for each job.

Who is Gufo for?

Developers and power users with an AMD Ryzen AI MAX+ 395 (Strix Halo) machine who want to run large open models locally on their own hardware.

What can I do with Gufo?

4 use cases so far, for example: Run Qwen3.8 27B as a private local API on Strix Halo; Point Codex at a local Qwen model through Gufo; Build Gufo from source with Nix and run diagnostics. Each has steps and a copy-paste agent prompt.

What do I need to run Gufo?

Linux x86-64 machine with AMD Strix Halo (gfx1151) and enough unified memory for the chosen model; Podman with rootless GPU device access to /dev/kfd and /dev/dri; Hugging Face CLI (hf) to download model weights; Several tens of GB of free disk space for model files.

What are alternatives to Gufo?

Other hot AI repos in Software and DevOps: Deep Infer, Harness2, Orca, Opencode, Graphify.

Is Gufo safe to run?

Deduped does not own, audit or vouch for it. Read the README and license, run it on your own systems first and review every command before you run it.