# AI use cases for Llam Ampere (JakeATX/llamAmpere) Repo: [JakeATX/llamAmpere](https://github.com/JakeATX/llamAmpere) | Use cases: 3 | Page: https://dedupedai.com/use-cases/jakeatx-llamampere/ llamAmpere is a C++ fork of llama.cpp tuned to run Qwen3.8-27B on Ampere NVIDIA cards such as the RTX 3090 and 3090 Ti, with a compressed KV cache and an MTP drafter. This file holds every use case of this repo. Each one starts with a "## " heading and ends with its URL. - Run a local 262K-context model server on a 24 GB card: https://dedupedai.com/use-cases/jakeatx-llamampere/run-local-qwen-server-24gb-card/ - Run 204,800-token context on a 12 GB card: https://dedupedai.com/use-cases/jakeatx-llamampere/run-long-context-model-on-12gb-card/ - Speed up repeat evals with a prebuilt prompt cache: https://dedupedai.com/use-cases/jakeatx-llamampere/reuse-prebuilt-cache-for-repeat-evals/ ## How to run a local 262K-context model server on a 24 GB card with Llam Ampere Repo: [JakeATX/llamAmpere](https://github.com/JakeATX/llamAmpere) | Difficulty: Intermediate | Time: 1 hour | Basis: From the README Build llama-server for an RTX 3090 and serve Qwen3.8-27B on a local OpenAI-compatible API. **Who it is for:** A developer with one RTX 3090 or 3090 Ti who wants a private, local model endpoint for coding or agent tools and needs long context. **The problem:** Running a 27B model with a very long context on a single consumer card usually needs heavy tuning, and the defaults are often too slow or too large for memory. **The outcome:** A llama-server process listening on http://127.0.0.1:8080 with a 262,144-token context, the MTP drafter and the prompt cache on by default. ### Steps 1. Run `git clone https://github.com/JakeATX/llamAmpere.git` and then `cd llamAmpere`. 2. Run `cmake -S . -B build-sm86 -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=86 -DLLAMA_BUILD_BORINGSSL=ON`. (inferred, check the README) 3. Run `cmake --build build-sm86 -j8 --target llama-server`. 4. Run `./build-sm86/bin/llama-server --hf-repo jakeatx/ATX-Swift-1.5-Qwen3.8-27B-Uncensored-IQ4_XS-M-GGUF --hf-file ATX-Swift-1.5-Qwen3.8-27B-Uncensored-IQ4_XS-M.gguf -c 262144 -ngl 99 -fa on -ctk tq5_0 -ctv turbo4 -b 4096 -ub 1024 --parallel 1`. (inferred, check the README) 5. Check that the server is up at http://127.0.0.1:8080 and send one chat request to its OpenAI-compatible API. ### Prompt to paste into your agent ```text In the llamAmpere repo at , build llama-server with the SM86 CMake settings and start it with the tq5_0/turbo4 command from the README. Send one chat request to http://127.0.0.1:8080 with the prompt , and report the reply text and the HTTP status. Verify that the server accepted a context size of 262144. ``` ### Caveat The README tests the build on Linux with CUDA 12.4 and a 24 GB NVIDIA card, and the model is an uncensored fine-tune, so review its outputs before use. Dual-use notice: only use this on software, sites and systems you own or are authorized to test. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/jakeatx-llamampere/run-local-qwen-server-24gb-card/ ## How to run 204,800-token context on a 12 GB card with Llam Ampere Repo: [JakeATX/llamAmpere](https://github.com/JakeATX/llamAmpere) | Difficulty: Advanced | Time: 1 hour | Basis: From the README Serve a 2.3 bpw Qwen3.8 model with SJ-KVaRN cache on an RTX 3060 12 GB or similar card. **Who it is for:** A developer with a 12 GB NVIDIA card (RTX 3060 12 GB, 3080 12 GB or 3080 Ti) who wants long-context coding, agentic or RAG work locally. **The problem:** Long context on a 12 GB card runs out of memory quickly, so most setups cap the context far below what agent or RAG jobs need. **The outcome:** A llama-server process with a 204,800-token context that the README reports at about 11 GB of card memory, with the MTP drafter enabled. ### Steps 1. Run `git clone https://github.com/JakeATX/llamAmpere.git` and then `cd llamAmpere`. 2. Run `cmake -S . -B build-sm86 -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=86 -DLLAMA_BUILD_BORINGSSL=ON`. (inferred, check the README) 3. Run `cmake --build build-sm86 -j8 --target llama-server`. 4. Run `LLAMA_MTP_DRAFT_COMPUTE_LEAN=1 ./build-sm86/bin/llama-server -hf jakeatx/ATX-Swift-1.5-Qwen3.8-27B-Uncensored-2.3bpw-MTP-GGUF -hff ATX-Swift-1.5-Qwen3.8-27B-Uncensored-2.3bpw-MTP.gguf -c 204800 --parallel 1 -ngl 99 -fa on -fit off -b 4096 -ub 512 --no-context-shift --jinja -ctk sjkvarn3 -ctv sjkvarn3 --sjkvarn-body-type sjkvarn4t --sjkvarn-sink 128 --sjkvarn-sink-type f16 --sjkvarn-staging-type tq6_0 --sjkvarn-tail 4096 --sjkvarn-tail-max 8192 --cache-type-s q8_0 --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0 --spec-draft-vocab-map docs/mtp-vocab/atx_65536.txt --spec-draft-type-k q8_0 --spec-draft-type-v q8_0 --spec-draft-window 8192 --temp 1.0 --top-k 20 --top-p 0.95 --min-p 0`. (inferred, check the README) 5. Watch GPU memory with your normal system monitor while a long prompt runs, and compare it with the README figure of about 12,052 MiB for the whole card. ### Prompt to paste into your agent ```text In the llamAmpere repo at , build llama-server and start the 2.3 bpw 12 GB command from the README on a 12 GB card. Send a prompt of about tokens to the local server and report the reply plus the peak GPU memory from nvidia-smi. Verify that the context setting is 204800. ``` ### Caveat The README measured the 12 GB setup on its own cards, and the command has many tuned flags, so a different driver or card may need the README's recommended settings doc instead. Dual-use notice: only use this on software, sites and systems you own or are authorized to test. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/jakeatx-llamampere/run-long-context-model-on-12gb-card/ ## How to speed up repeat evals with a prebuilt prompt cache with Llam Ampere Repo: [JakeATX/llamAmpere](https://github.com/JakeATX/llamAmpere) | Difficulty: Advanced | Time: 1.5 hours | Basis: From the README Point llama-server at a read-only prompt cache folder so long eval prompts skip re-reading. **Who it is for:** An ML engineer or researcher who runs the same long eval or benchmark prompts many times against Swift 1.5 on Llam Ampere. **The problem:** Each eval run re-reads the same long prompts, which wastes time on every repeat run. **The outcome:** Eval prompts load from a prebuilt folder, and at temperature 0 the output matches the uncached run according to the README. ### Steps 1. Complete the build in the first use case so `./build-sm86/bin/llama-server` exists in the Llam Ampere folder. 2. Download a ready-made cache folder from https://huggingface.co/datasets/jakeatx/llamampere-evalcache to a local directory. (inferred, check the README) 3. Start the server with the Swift 1.5 command from the 24 GB use case and add `--eval-cache `. (inferred, check the README) 4. Run the same eval prompts twice at temperature 0 and compare the outputs and the timing. 5. Check the server log to confirm that the eval prompts loaded from the cache folder. ### Prompt to paste into your agent ```text In the llamAmpere repo at , start llama-server with the Swift 1.5 model and `--eval-cache `. Run the eval prompts in twice at temperature 0 and report the time of each run and whether the two outputs match. Verify that the second run is faster and that the outputs are identical. ``` ### Caveat The README describes --eval-cache as an opt-in, read-only cache for eval and benchmark prompts, and it does not give the exact command syntax, so check the release notes at docs/llamampere-v0.5.5/RELEASE_NOTES.md before running it. Dual-use notice: only use this on software, sites and systems you own or are authorized to test. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/jakeatx-llamampere/reuse-prebuilt-cache-for-repeat-evals/