DEDUPED Hot AI repos. Real use cases. What people build. Get the Monday 5

How to ask many questions about one long document quickly with MLX LM

Cache a long prompt once with mlx_lm.cache_prompt and reuse it for each new question.

Short answer: Install with pip install mlx-lm; then Cache the document: pipe prompt.txt into mlx_lm.cache_prompt with --model, --prompt - and --prompt-cache-file mistral_prompt.safetensors, as in the README; ask a question with mlx_lm.generate using --prompt-cache-file mistral_prompt.safetensors and your prompt; the model is read from the cache. All steps.

Who it is forSomeone querying the same long report or codebase dump many times on a Mac.

The problemRe-reading a long context for every question is slow and memory hungry.

The outcomeFast follow-up answers that reuse a saved prompt cache file.

What you need first

  • A Mac with Apple silicon
  • Python with pip or conda
  • macOS 15 or later to speed up models that are large relative to RAM
  • Optional: a Hugging Face account to upload converted models

Steps

  1. Install with pip install mlx-lm
  2. Cache the document: pipe prompt.txt into mlx_lm.cache_prompt with --model, --prompt - and --prompt-cache-file mistral_prompt.safetensors, as in the README
  3. Ask a question with mlx_lm.generate using --prompt-cache-file mistral_prompt.safetensors and your prompt; the model is read from the cache
  4. Lower peak memory while reading long prompts with --prefill-step-size
  5. Use the rotating KV cache with --max-kv-size for very long generations

Prompt for your agent

Using ml-explore/mlx-lm, cache <document> with mlx_lm.cache_prompt into a safetensors file, then answer these questions one by one with mlx_lm.generate --prompt-cache-file: <questions>.

Caveat. Smaller --max-kv-size values save RAM but lower quality, as the README warns.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for ml-explore/mlx-lm and "Ask many questions about one long document quickly"

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer