DEDUPED Hot AI repos. Real use cases. What people build. Get the Monday 5

How to chat with an open LLM locally on your Mac with MLX LM

Install MLX LM and generate text or open a chat REPL with any MLX-compatible model from the Hub.

Short answer: Install with pip install mlx-lm or conda install -c conda-forge mlx-lm; then generate text with mlx_lm.generate --prompt "How tall is Mt Everest?"; start a chat REPL that keeps context for its lifetime with mlx_lm.chat. All steps.

  • Top pick
  • Software and DevOps
  • Beginner
  • 15 minutes
  • From the README
  • Claude Code
  • Codex
  • Cursor
  • Gemini CLI

Who it is forA Mac user who wants a private local assistant without a cloud API.

The problemCloud chat sends your data off the machine and costs money per token.

The outcomeA local chat REPL running a quantized model on Apple silicon.

What you need first

  • A Mac with Apple silicon
  • Python with pip or conda
  • macOS 15 or later to speed up models that are large relative to RAM
  • Optional: a Hugging Face account to upload converted models

Steps

  1. Install with pip install mlx-lm or conda install -c conda-forge mlx-lm
  2. Generate text with mlx_lm.generate --prompt "How tall is Mt Everest?"
  3. Start a chat REPL that keeps context for its lifetime with mlx_lm.chat
  4. Pick another model with --model, for example mlx_lm.generate --model mistralai/Mistral-7B-Instruct-v0.3 --prompt "hello"
  5. See all options with mlx_lm.generate -h

Prompt for your agent

Using ml-explore/mlx-lm on my Mac, install it, run mlx_lm.chat with a model from mlx-community that fits in <RAM> GB, and tell me the tokens per second it reports.

Caveat. The default model is mlx-community/Llama-3.2-3B-Instruct-4bit; bigger models need more unified memory.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for ml-explore/mlx-lm and "Chat with an open LLM locally on your Mac"

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer