DEDUPED Hot AI repos, turned into use cases Get the Monday 5

How to chat with local models from the terminal with LocalAI

Run a model in one terminal and chat with it in another, switching models inside the session.

Short answer: Install LocalAI (Docker or the macOS app); then in terminal 1 run local-ai run llama-3.2-1b-instruct:q4_k_m; in terminal 2 run local-ai chat --model llama-3.2-1b-instruct:q4_k_m. All steps.

Who it is forA developer who wants quick private answers without opening a browser or paying per token.

The problemTrying different local models usually means juggling several tools and configs.

The outcomeA terminal chat against any installed model, with commands to list and switch models.

What you need first

  • Docker or Podman, or the macOS app (not signed by Apple)
  • Optional GPU: NVIDIA, AMD, Intel, Apple Silicon or Vulkan; CPU-only works
  • Disk space for the models you pull

Steps

  1. Install LocalAI (Docker or the macOS app)
  2. In terminal 1 run local-ai run llama-3.2-1b-instruct:q4_k_m
  3. In terminal 2 run local-ai chat --model llama-3.2-1b-instruct:q4_k_m
  4. Inside the session use /models to list installed models and /model to switch
  5. Measure latency and throughput with local-ai benchmark

Prompt for your agent

Using mudler/LocalAI, run two different small models, chat with each using local-ai chat on the same three questions, run local-ai benchmark, and recommend which model to keep for <use>.

Caveat.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for mudler/LocalAI and "Chat with local models from the terminal"

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer