DEDUPED Hot AI repos, turned into use cases Get the Monday 5

How to run separate chat sessions on one loaded model with Go Llama

Load a model once and give each conversation its own context and KV cache.

Short answer: Call llama.New and LoadModel once at startup and keep the model open; then call model.NewContext with llama.ContextParams once for each conversation; call ctx.Generate for each user's prompt against that user's own context. All steps.

Who it is forA developer serving several users or chat sessions from one machine who wants to avoid loading the weights repeatedly.

The problemLoading a separate model per user would multiply memory use and startup time.

The outcomeOne shared model serves independent conversations, each with its own context.

What you need first

  • A Go toolchain installed on your machine
  • A GGUF model file, such as model.gguf
  • Enough memory for the model weights and KV cache, since the README notes wasm32 caps linear memory at 4 GiB

Steps

  1. Call llama.New and LoadModel once at startup and keep the model open.
  2. Call model.NewContext with llama.ContextParams once for each conversation.
  3. Call ctx.Generate for each user's prompt against that user's own context.
  4. Call ctx.Close and model.Close when a session ends or the service shuts down.

Prompt for your agent

In the Go service at <project-path>, load <model-file>.gguf once with goccy/go-llama and create one context per session ID. Expected output: two sessions returning responses that each reflect only their own prompt history. Verify that closing one context does not affect generation in the other.

Caveat. The README says contexts keep their own KV cache but does not document a built-in session manager, so you must track which context belongs to which user yourself.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for goccy/go-llama and "Run separate chat sessions on one loaded model"

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer