# AI use cases for Kyojin (Yamz-Labs/kyojin) Repo: [Yamz-Labs/kyojin](https://github.com/Yamz-Labs/kyojin) | Use cases: 5 | Page: https://dedupedai.com/use-cases/yamz-labs-kyojin/ Kyojin is an open source inference engine, built on ExLlamaV3, that runs large GLM, Qwen and MiMo models on one AMD Strix Halo mini PC with 128 GB of memory. This file holds every use case of this repo. Each one starts with a "## " heading and ends with its URL. - Run a local OpenAI-style LLM server on AMD: https://dedupedai.com/use-cases/yamz-labs-kyojin/run-local-openai-compatible-server/ - Read a 500K-token document in one prompt: https://dedupedai.com/use-cases/yamz-labs-kyojin/analyze-500k-token-documents-locally/ - Benchmark a Strix Halo machine and share results: https://dedupedai.com/use-cases/yamz-labs-kyojin/benchmark-and-share-strix-halo-results/ - Scrape server metrics from a local model host: https://dedupedai.com/use-cases/yamz-labs-kyojin/monitor-local-inference-server-metrics/ - Load a less-restricted model variant locally: https://dedupedai.com/use-cases/yamz-labs-kyojin/load-less-restricted-model-for-testing/ ## How to run a local OpenAI-style LLM server on AMD with Kyojin Repo: [Yamz-Labs/kyojin](https://github.com/Yamz-Labs/kyojin) | Difficulty: Intermediate | Time: 2 hours | Basis: From the README Serve a 95 GB Qwen model on one 128 GB AMD mini PC through an OpenAI-compatible API on port 8000. **Who it is for:** A developer or small team that wants to point existing OpenAI API client code at a model hosted on their own AMD Strix Halo machine instead of a cloud service. **The problem:** Cloud model APIs send data off the machine and bill per request, and most large models do not fit on a single consumer box. **The outcome:** A local server on port 8000 that answers OpenAI API requests using a 300B-class MoE model class on one machine. ### Steps 1. Install ROCm torch and build Kyojin once by following doc/install.md. 2. `git clone https://github.com/Yamz-Labs/kyojin && cd kyojin` 3. `hf download yamz-labs/Qwen3.8-Flash-Next-EXL3-Yamz --local-dir ./qwen-pack` 4. `python tools/qwen/serve.py --model ./qwen-pack --port 8000 -c 131072` 5. Point your OpenAI API client at port 8000 on the machine and send a test chat request. ### Prompt to paste into your agent ```text In the Yamz-Labs/kyojin repo, follow doc/install.md, then download the Qwen3.8-Flash-Next pack and start tools/qwen/serve.py on port 8000 on . Expected output: the server starts and answers one OpenAI API chat request with text. Verify that the response JSON includes a choices array before reporting success. ``` ### Caveat The README says the software targets AMD Strix Halo with ROCm (gfx1151), so it will not run on other hardware as documented. Dual-use notice: only use this on software, sites and systems you own or are authorized to test. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/yamz-labs-kyojin/run-local-openai-compatible-server/ ## How to read a 500K-token document in one prompt with Kyojin Repo: [Yamz-Labs/kyojin](https://github.com/Yamz-Labs/kyojin) | Difficulty: Advanced | Time: 3 hours | Basis: From the README Load a very long report or codebase dump and query it locally with context up to 500K tokens. **Who it is for:** An analyst, lawyer or researcher with a very long document set who cannot send the text to a hosted service. **The problem:** Standard context windows force splitting long documents into chunks, which loses cross-references between sections. **The outcome:** A local server that accepts one prompt of about 500K tokens and answers questions about the whole text. ### Steps 1. Install ROCm torch and build Kyojin per doc/install.md. 2. `hf download yamz-labs/Qwen3.8-Flash-Next-EXL3-Yamz --local-dir ./qwen-pack` 3. `python tools/qwen/serve.py --model ./qwen-pack --port 8000 -c 524288 --cache-bits 8` (inferred, check the README) 4. Send the full document as one prompt to the OpenAI API on port 8000 and ask a specific question about it. 5. Check the answer against the source text for two facts placed at different depths. ### Prompt to paste into your agent ```text In the Yamz-Labs/kyojin repo, start tools/qwen/serve.py with -c 524288 and --cache-bits 8 for the Qwen pack, then send as a single prompt. Expected output: an answer that cites facts from the document. Verify by checking two facts from the start and end of the text against the source before reporting. ``` ### Caveat The README says the 500K setting needs -c 524288 at launch and the Qwen run also needs --cache-bits 8, so longer contexts require more memory and tuning. Dual-use notice: only use this on software, sites and systems you own or are authorized to test. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/yamz-labs-kyojin/analyze-500k-token-documents-locally/ ## How to benchmark a Strix Halo machine and share results with Kyojin Repo: [Yamz-Labs/kyojin](https://github.com/Yamz-Labs/kyojin) | Difficulty: Intermediate | Time: 1 hour | Basis: From the README Measure prefill and speculative decode speed on your own 128 GB AMD box and export the numbers as JSON. **Who it is for:** A hardware reviewer or tester with a Ryzen AI Max+ 395 machine who wants to compare its speed with published Kyojin figures. **The problem:** Published speed numbers come from one machine, and buyers want to know what their own unit delivers. **The outcome:** A benchmark report for your machine that you can compare with the tables in doc/benchmarks.md and submit as a report. ### Steps 1. Install ROCm torch and build Kyojin per doc/install.md. 2. Read doc/benchmarks.md to see how the figures are measured. 3. `tools/bench.sh --json` 4. Compare your prefill and speculative decode numbers with the tables in doc/benchmarks.md. 5. Send the result through the reporting path in CONTRIBUTING.md. ### Prompt to paste into your agent ```text In the Yamz-Labs/kyojin repo, run tools/bench.sh --json on after reading doc/benchmarks.md. Expected output: a JSON benchmark result with prefill and speculative decode tokens per second. Verify that the figures are labeled with the context length and model used before comparing them to the README tables. ``` ### Caveat The README says the published numbers come from one Ryzen AI Max+ 395 with 128 GB, so your results will differ with other hardware and settings. Dual-use notice: only use this on software, sites and systems you own or are authorized to test. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/yamz-labs-kyojin/benchmark-and-share-strix-halo-results/ ## How to scrape server metrics from a local model host with Kyojin Repo: [Yamz-Labs/kyojin](https://github.com/Yamz-Labs/kyojin) | Difficulty: Advanced | Time: 1 hour | Basis: From the README Expose Kyojin's Prometheus metrics endpoint so a monitoring stack can track a local model server. **Who it is for:** A homelab or small ops owner who runs the model server on a mini PC and wants its health and load in an existing dashboard. **The problem:** A local model server gives no visibility into load or throughput unless something collects its numbers. **The outcome:** A Prometheus setup that reads the server's /metrics endpoint on port 8000. ### Steps 1. Start the server per doc/models.md, for example with `python tools/qwen/serve.py --model ./qwen-pack --port 8000 -c 131072`. 2. Check that the endpoint responds at /metrics on the host. 3. Add a Prometheus scrape job that targets the host on port 8000 at path /metrics. (inferred, check the README) 4. Open the Prometheus UI and confirm the target shows as up. (inferred, check the README) ### Prompt to paste into your agent ```text In the Yamz-Labs/kyojin repo, start the Qwen server on port 8000 and confirm the Prometheus /metrics endpoint responds on . Expected output: a plain text metrics page from the server. Verify that the response contains metric lines before adding it as a scrape target. ``` ### Caveat The README credits the /metrics endpoint as merged for GLM and MiMo, so confirm it is available for the model you run before building dashboards. Dual-use notice: only use this on software, sites and systems you own or are authorized to test. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/yamz-labs-kyojin/monitor-local-inference-server-metrics/ ## How to load a less-restricted model variant locally with Kyojin Repo: [Yamz-Labs/kyojin](https://github.com/Yamz-Labs/kyojin) | Difficulty: Advanced | Time: 3 hours | Basis: From the README Run an optional uncensored mode on GLM, MiMo or the Qwen preset for controlled red-team and research tests. **Who it is for:** A security researcher or AI safety tester who needs a model that refuses fewer requests to study its behavior on their own machine. **The problem:** Hosted models refuse many requests, which blocks some legitimate adversarial testing and evaluation work. **The outcome:** A local model load with the uncensored mode applied at load time, with the original weights left unchanged. ### Steps 1. Read doc/refusal_hook.md to understand how the mode works. 2. Download the uncensored pack for GLM-5.3-Flash or MiMo-V2.6-Flash from the links in the README, or enable the preset in the Qwen3.8-Flash-Next pack. 3. Follow the serve commands in doc/models.md for that model. 4. Run only prompts that are part of an approved test plan and log every request and response. ### Prompt to paste into your agent ```text In the Yamz-Labs/kyojin repo, read doc/refusal_hook.md and load the uncensored variant of following doc/models.md. Expected output: the server starts with the mode applied and returns responses to a scoped test prompt. Verify that the test is covered by an approved plan and that all requests are logged before running it. ``` ### Caveat The README says the mode is off by default for the Qwen preset and that the other packs are first versions, so behavior may change between releases. Risk, reverse engineering: The uncensored mode is designed to make the model refuse far less, so outputs must be kept inside an approved test plan and not used to generate harmful content. Dual-use notice: only use this on software, sites and systems you own or are authorized to test. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/yamz-labs-kyojin/load-less-restricted-model-for-testing/