DEDUPED Hot AI repos, turned into use cases Get 5 a day

What you can do with Llam Ampere

llamAmpere is a C++ fork of llama.cpp tuned to run Qwen3.8-27B on Ampere NVIDIA cards such as the RTX 3090 and 3090 Ti, with a compressed KV cache and an MTP drafter.

Trended earlier

JakeATX/llamAmpere on GitHub

Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk.

Authorization notice. This tool is dual-use. Only use it on software, sites and systems you own or have explicit permission to test, and follow the relevant licenses and terms of service.

What do you want to do?

3 use cases

  • Run a local 262K-context model server on a 24 GB card

    Build llama-server for an RTX 3090 and serve Qwen3.8-27B on a local OpenAI-compatible API.

    View prompt
    In the llamAmpere repo at <repo path>, build llama-server with the SM86 CMake settings and start it with the tq5_0/turbo4 command from the README. Send one chat request to http://127.0.0.1:8080 with the prompt <test prompt>, and report the reply text and the HTTP status. Verify that the server accepted a context size of 262144.

  • Run 204,800-token context on a 12 GB card

    Serve a 2.3 bpw Qwen3.8 model with SJ-KVaRN cache on an RTX 3060 12 GB or similar card.

    View prompt
    In the llamAmpere repo at <repo path>, build llama-server and start the 2.3 bpw 12 GB command from the README on a 12 GB card. Send a prompt of about <token count> tokens to the local server and report the reply plus the peak GPU memory from nvidia-smi. Verify that the context setting is 204800.

  • Speed up repeat evals with a prebuilt prompt cache

    Point llama-server at a read-only prompt cache folder so long eval prompts skip re-reading.

    View prompt
    In the llamAmpere repo at <repo path>, start llama-server with the Swift 1.5 model and `--eval-cache <cache directory>`. Run the eval prompts in <eval file> twice at temperature 0 and report the time of each run and whether the two outputs match. Verify that the second run is faster and that the outputs are identical.

How to use Llam Ampere

  1. Check you have what it needs: Linux machine with an NVIDIA card (RTX 3090 or 3090 Ti with 24 GB for the main path, or a 12 GB card for the low-memory path); CUDA toolkit (the README tested with 12.4), CMake, git and a C++ compiler; Disk space and network access for the model download: 15.6 GB for the 24 GB path, 9.0 GB for the 12 GB path.
  2. Open JakeATX/llamAmpere on GitHub and read the README and license.
  3. Run git clone https://github.com/JakeATX/llamAmpere.git and then cd llamAmpere.
  4. Run cmake -S . -B build-sm86 -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=86 -DLLAMA_BUILD_BORINGSSL=ON. (inferred, check the README)
  5. Run cmake --build build-sm86 -j8 --target llama-server.
  6. Run ./build-sm86/bin/llama-server --hf-repo jakeatx/ATX-Swift-1.5-Qwen3.8-27B-Uncensored-IQ4_XS-M-GGUF --hf-file ATX-Swift-1.5-Qwen3.8-27B-Uncensored-IQ4_XS-M.gguf -c 262144 -ngl 99 -fa on -ctk tq5_0 -ctv turbo4 -b 4096 -ub 1024 --parallel 1. (inferred, check the README)

These steps come from our top use case, Run a local 262K-context model server on a 24 GB card, which also has a copy-paste agent prompt.

Llam Ampere alternatives · Best AI repos for Software and DevOps

About this repo

  • Who it is forBuilt for people comfortable with a Linux terminal, CMake and CUDA who want a local OpenAI-compatible model server on a single NVIDIA card.
  • Why it is hot
  • LanguageC++
  • Sourcegithub.com/JakeATX/llamAmpere · Back to Hot repos

Requirements

  • Linux machine with an NVIDIA card (RTX 3090 or 3090 Ti with 24 GB for the main path, or a 12 GB card for the low-memory path)
  • CUDA toolkit (the README tested with 12.4), CMake, git and a C++ compiler
  • Disk space and network access for the model download: 15.6 GB for the 24 GB path, 9.0 GB for the 12 GB path
  • Recommended: OpenSSL development files are not needed if you use the -DLLAMA_BUILD_BORINGSSL=ON build flag, which lets -hf download the model
Maintain Llam Ampere? Add the Featured on Deduped badge

Paste this into your README:

[![Featured on Deduped](https://dedupedai.com/badge/jakeatx-llamampere.svg)](https://dedupedai.com/use-cases/jakeatx-llamampere/?ref=badge)

More badge styles and HTML

Questions about Llam Ampere

What is Llam Ampere?

llamAmpere is a C++ fork of llama.cpp tuned to run Qwen3.8-27B on Ampere NVIDIA cards such as the RTX 3090 and 3090 Ti, with a compressed KV cache and an MTP drafter. It is an open-source project on GitHub: https://github.com/JakeATX/llamAmpere.

Who is Llam Ampere for?

Built for people comfortable with a Linux terminal, CMake and CUDA who want a local OpenAI-compatible model server on a single NVIDIA card.

What can I do with Llam Ampere?

3 use cases so far, for example: Run a local 262K-context model server on a 24 GB card; Run 204,800-token context on a 12 GB card; Speed up repeat evals with a prebuilt prompt cache. Each has steps and a copy-paste agent prompt.

What do I need to run Llam Ampere?

Linux machine with an NVIDIA card (RTX 3090 or 3090 Ti with 24 GB for the main path, or a 12 GB card for the low-memory path); CUDA toolkit (the README tested with 12.4), CMake, git and a C++ compiler; Disk space and network access for the model download: 15.6 GB for the 24 GB path, 9.0 GB for the 12 GB path; Recommended: OpenSSL development files are not needed if you use the -DLLAMA_BUILD_BORINGSSL=ON build flag, which lets -hf download the model.

What are alternatives to Llam Ampere?

Other hot AI repos in Software and DevOps: Harness2, iFixAi, Orca, Diagram Design, AI Memory.

Is Llam Ampere safe to run?

Deduped does not own, audit or vouch for it. Read the README and license, run it on your own systems first and review every command before you run it.