DEDUPED Hot AI repos, turned into use cases Get the Monday 5

How to run a GGUF model from Hugging Face with llama.cpp

Install llama.cpp and chat with a GGUF model pulled straight from Hugging Face.

Remote script. The quick install pipes a script from llama.app into your shell, so review it first or use the release binaries.

Who it is forA developer who wants to try a local model without Python environments.

The problemSetting up local inference often means fighting dependencies.

The outcomeA local chat session with a downloaded model.

What you need first

  • macOS, Linux or Windows
  • Enough RAM or VRAM for the chosen GGUF model

Steps

  1. Install with curl -LsSf https://llama.app/install.sh | sh (or irm https://llama.app/install.ps1 | iex on Windows)
  2. Run llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF
  3. Chat in the terminal
  4. Try a larger GGUF model if your hardware allows

Prompt for your agent

Install ggml-org/llama.cpp and run `llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF`. Ask it <question> and report tokens per second shown, confirming the model ran locally.

Caveat. Pre-built binaries and Docker are alternatives if you prefer not to run the install script.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for ggml-org/llama.cpp and "Run an open model locally in one command"

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer