DEDUPED Hot AI repos, turned into use cases Get 5 a day

What you can do with Lithos Metal

lithos-metal is an open-source LLM inference engine that generates Metal megakernels and serves local models on Apple silicon Macs.

Trended earlier

lithos-ai/lithos-metal on GitHub

Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk.

What do you want to do?

4 use cases

  • Run a local model as the backend for a coding agent

    Serve Qwen3.8-27B on your Mac and point a coding agent at it without sending code to a hosted API.

    View prompt
    Using the lithos-metal repo at https://github.com/lithos-ai/lithos-metal, start the server with `lithos-metal serve --model nvidia/Qwen3.8-27B-NVFP4` on this Mac, then run `lithos-metal <client>` where <client> is the coding agent you want to use. Report the served model name and the first reply to a one-line test prompt. Verify that http://127.0.0.1:8000 answers before reporting success.

  • Call the local model from scripts over HTTP

    Send chat requests to the local server from your own code, with optional streaming.

    View prompt
    Using the lithos-metal repo at https://github.com/lithos-ai/lithos-metal, with the server running via `lithos-metal serve --model nvidia/Qwen3.8-27B-NVFP4`, write a short script that sends <your prompt> to http://127.0.0.1:8000/v1/chat/completions and prints the reply. Expected output is the model's text reply. Verify the script receives an HTTP 200 response before reporting success.

  • Compare speculative and target-only generation

    Run the same model with DSpark drafting on and off to see how the setting changes output speed.

    View prompt
    Using the lithos-metal repo at https://github.com/lithos-ai/lithos-metal, run the same fixed prompt against `lithos-metal serve --model nvidia/Qwen3.8-27B-NVFP4` and again with `--no-draft`. Report wall-clock time and generated token count for each run in a two-row table. Verify both runs used the same prompt and max_completion_tokens before comparing.

  • Build Lithos Metal from source for development

    Compile the native runtime from the repo and install an editable copy for changes.

    View prompt
    Using the lithos-metal repo at https://github.com/lithos-ai/lithos-metal, clone it, create a Python 3.12 virtual environment, and run `pip install -e '.[serve,dev]'`. Expected output is a successful install and a version string from `lithos-metal --version`. Verify the command prints a version before reporting success.

How to use Lithos Metal

  1. Check you have what it needs: An Apple silicon Mac running macOS 26 or later; Enough unified memory for the target model, draft model, and context (the README tests Qwen models on a 40-core M5 Max with 48 GB); Homebrew, or Xcode Command Line Tools and Python 3.12 for a source build.
  2. Open lithos-ai/lithos-metal on GitHub and read the README and license.
  3. Install the precompiled package with brew install lithos-ai/tap/lithos-metal.
  4. Start the server with lithos-metal serve --model nvidia/Qwen3.8-27B-NVFP4 and wait for compilation and warmup to finish.
  5. Install your client separately, for example Claude Code or Codex, in a second terminal.
  6. Launch the client through Lithos Metal with lithos-metal claude or lithos-metal codex.

These steps come from our top use case, Run a local model as the backend for a coding agent, which also has a copy-paste agent prompt.

Lithos Metal alternatives · Best AI repos for Software and DevOps

About this repo

  • Who it is forDevelopers and engineers on an Apple silicon Mac who want a fast local model server for coding agents or apps that call an OpenAI-compatible API.
  • Why it is hot
  • LanguagePython
  • Sourcegithub.com/lithos-ai/lithos-metal · Back to Hot repos

Requirements

  • An Apple silicon Mac running macOS 26 or later
  • Enough unified memory for the target model, draft model, and context (the README tests Qwen models on a 40-core M5 Max with 48 GB)
  • Homebrew, or Xcode Command Line Tools and Python 3.12 for a source build
  • Client tools such as opencode, claude, codex, or hermes installed separately, if you use the launchers
Maintain Lithos Metal? Add the Featured on Deduped badge

Paste this into your README:

[![Featured on Deduped](https://dedupedai.com/badge/lithos-ai-lithos-metal.svg)](https://dedupedai.com/use-cases/lithos-ai-lithos-metal/?ref=badge)

More badge styles and HTML

Questions about Lithos Metal

What is Lithos Metal?

lithos-metal is an open-source LLM inference engine that generates Metal megakernels and serves local models on Apple silicon Macs. It is an open-source project on GitHub: https://github.com/lithos-ai/lithos-metal.

Who is Lithos Metal for?

Developers and engineers on an Apple silicon Mac who want a fast local model server for coding agents or apps that call an OpenAI-compatible API.

What can I do with Lithos Metal?

4 use cases so far, for example: Run a local model as the backend for a coding agent; Call the local model from scripts over HTTP; Compare speculative and target-only generation. Each has steps and a copy-paste agent prompt.

What do I need to run Lithos Metal?

An Apple silicon Mac running macOS 26 or later; Enough unified memory for the target model, draft model, and context (the README tests Qwen models on a 40-core M5 Max with 48 GB); Homebrew, or Xcode Command Line Tools and Python 3.12 for a source build; Client tools such as opencode, claude, codex, or hermes installed separately, if you use the launchers.

What are alternatives to Lithos Metal?

Other hot AI repos in Software and DevOps: Harness2, iFixAi, Orca, Diagram Design, AI Memory.

Is Lithos Metal safe to run?

Deduped does not own, audit or vouch for it. Read the README and license, run it on your own systems first and review every command before you run it.