DEDUPED Hot AI repos, turned into use cases Get the Monday 5

How to run a third-party GGUF model with limited file and memory access with Go Llama

Restrict the model to one directory and cap its memory using llama.New options.

Short answer: Place the model file in one dedicated directory such as /srv/models; then Pass llama.WithPreopenDir with that directory to llama.New; set a memory limit with llama.WithMaxMemory to cap growth. All steps.

Who it is forA security-minded engineer who must run a model file from an outside source on a shared server.

The problemAn unknown model file should not be able to see the whole filesystem or grow memory without limit.

The outcomeThe guest can read only the chosen model directory, and an oversized model fails inside the sandbox.

What you need first

  • A Go toolchain installed on your machine
  • A GGUF model file, such as model.gguf
  • Enough memory for the model weights and KV cache, since the README notes wasm32 caps linear memory at 4 GiB

Steps

  1. Place the model file in one dedicated directory such as /srv/models.
  2. Pass llama.WithPreopenDir with that directory to llama.New.
  3. Set a memory limit with llama.WithMaxMemory to cap growth.
  4. Run go run . and confirm an oversized model fails with an error instead of growing the host process. (inferred, check the README)

Prompt for your agent

In the Go project at <project-path>, create a llama.New instance with WithPreopenDir set to /srv/models and WithMaxMemory set to a limit you choose, then load <model-file>.gguf. Expected output: a successful load for a small model and an error for a model larger than the cap. Verify that the program cannot open files outside /srv/models.

Caveat. The README describes these sandbox options, but it does not say how strictly the limits are enforced, so test them against your own threat model before trusting them.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for goccy/go-llama and "Run a third-party GGUF model with limited file and memory access"

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer