# AI use cases for Go Llama (goccy/go-llama) Repo: [goccy/go-llama](https://github.com/goccy/go-llama) | Use cases: 4 | Page: https://dedupedai.com/use-cases/goccy-go-llama/ go-llama is a Go library that runs GGUF language models through llama.cpp compiled to WebAssembly and translated to Go, with no cgo or shared library needed. This file holds every use case of this repo. Each one starts with a "## " heading and ends with its URL. - Add local GGUF text generation to a Go program: https://dedupedai.com/use-cases/goccy-go-llama/embed-gguf-model-in-go-service/ - Show model output token by token as it generates: https://dedupedai.com/use-cases/goccy-go-llama/stream-tokens-to-live-output/ - Run separate chat sessions on one loaded model: https://dedupedai.com/use-cases/goccy-go-llama/serve-many-conversations-from-one-model/ - Run a third-party GGUF model with limited file and memory access: https://dedupedai.com/use-cases/goccy-go-llama/run-untrusted-gguf-in-sandbox/ ## How to add local GGUF text generation to a Go program with Go Llama Repo: [goccy/go-llama](https://github.com/goccy/go-llama) | Difficulty: Intermediate | Time: 1 hour | Basis: From the README Call a local GGUF model from Go code and get generated text back, with no cgo. **Who it is for:** A Go backend developer who needs text generation in a service and wants to avoid cgo or a separate inference server. **The problem:** Calling a hosted model API adds a network dependency, and cgo-based bindings complicate builds and cross-compiling. **The outcome:** A Go program that loads a GGUF file, generates text from a prompt, and prints the result. ### Steps 1. Install a Go toolchain and download a GGUF model file to your machine. 2. Run `go get github.com/goccy/go-llama` in your Go module. (inferred, check the README) 3. Copy the README example into main.go and set the LoadModel path to your model file. 4. Run `CGO_ENABLED=0 go run .` to build and run without cgo. (inferred, check the README) ### Prompt to paste into your agent ```text In the Go module at , add the goccy/go-llama package and use the README example to load .gguf and generate text from the prompt "Once upon a time". Expected output: the generated text printed to stdout. Verify that the program exits without error when built with CGO_ENABLED=0. ``` ### Caveat The README example uses a model file named model.gguf, and generation speed and memory use depend on the model size and quantization you choose. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/goccy-go-llama/embed-gguf-model-in-go-service/ ## How to show model output token by token as it generates with Go Llama Repo: [goccy/go-llama](https://github.com/goccy/go-llama) | Difficulty: Intermediate | Time: 1 hour | Basis: From the README Use Context.Stream to print or push each piece of text while the model is still generating. **Who it is for:** A developer building a chat screen, terminal tool, or log viewer who wants output to appear as it is produced. **The problem:** Waiting for a full response before showing anything makes long answers feel slow and hides progress. **The outcome:** Text appears incrementally, and a running generation can be stopped from another goroutine with Context.Interrupt. ### Steps 1. Load a GGUF model and create a context as shown in the README example. 2. Call Context.Stream instead of Generate and write each callback piece to your output. 3. Run a second goroutine that calls Context.Interrupt to test stopping a long generation. 4. Run `go run .` and watch text print as it decodes. (inferred, check the README) ### Prompt to paste into your agent ```text In the Go project at , use goccy/go-llama's Context.Stream to print each generated piece for the prompt "" from .gguf. Expected output: text that appears in pieces before the program finishes. Verify that calling Context.Interrupt from another goroutine stops the generation. ``` ### Caveat The README describes the Stream callback and Interrupt method but does not show their full signatures, so check the package documentation on pkg.go.dev before relying on exact parameter types. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/goccy-go-llama/stream-tokens-to-live-output/ ## How to run separate chat sessions on one loaded model with Go Llama Repo: [goccy/go-llama](https://github.com/goccy/go-llama) | Difficulty: Advanced | Time: 2 hours | Basis: From the README Load a model once and give each conversation its own context and KV cache. **Who it is for:** A developer serving several users or chat sessions from one machine who wants to avoid loading the weights repeatedly. **The problem:** Loading a separate model per user would multiply memory use and startup time. **The outcome:** One shared model serves independent conversations, each with its own context. ### Steps 1. Call llama.New and LoadModel once at startup and keep the model open. 2. Call model.NewContext with llama.ContextParams once for each conversation. 3. Call ctx.Generate for each user's prompt against that user's own context. 4. Call ctx.Close and model.Close when a session ends or the service shuts down. ### Prompt to paste into your agent ```text In the Go service at , load .gguf once with goccy/go-llama and create one context per session ID. Expected output: two sessions returning responses that each reflect only their own prompt history. Verify that closing one context does not affect generation in the other. ``` ### Caveat The README says contexts keep their own KV cache but does not document a built-in session manager, so you must track which context belongs to which user yourself. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/goccy-go-llama/serve-many-conversations-from-one-model/ ## How to run a third-party GGUF model with limited file and memory access with Go Llama Repo: [goccy/go-llama](https://github.com/goccy/go-llama) | Difficulty: Advanced | Time: 1 hour | Basis: From the README Restrict the model to one directory and cap its memory using llama.New options. **Who it is for:** A security-minded engineer who must run a model file from an outside source on a shared server. **The problem:** An unknown model file should not be able to see the whole filesystem or grow memory without limit. **The outcome:** The guest can read only the chosen model directory, and an oversized model fails inside the sandbox. ### Steps 1. Place the model file in one dedicated directory such as /srv/models. 2. Pass llama.WithPreopenDir with that directory to llama.New. 3. Set a memory limit with llama.WithMaxMemory to cap growth. 4. Run `go run .` and confirm an oversized model fails with an error instead of growing the host process. (inferred, check the README) ### Prompt to paste into your agent ```text In the Go project at , create a llama.New instance with WithPreopenDir set to /srv/models and WithMaxMemory set to a limit you choose, then load .gguf. Expected output: a successful load for a small model and an error for a model larger than the cap. Verify that the program cannot open files outside /srv/models. ``` ### Caveat The README describes these sandbox options, but it does not say how strictly the limits are enforced, so test them against your own threat model before trusting them. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/goccy-go-llama/run-untrusted-gguf-in-sandbox/