DEDUPED Hot AI repos. Real use cases. What people build. Get the Monday 5

How to run Gemma on your own machine from the terminal with LiteRT LM

Install the LiteRT LM CLI with uv and chat with a Gemma model without writing any code.

Short answer: Install the CLI with uv tool install litert-lm; then run litert-lm run with --from-huggingface-repo pointing at a model repo such as google/gemma-3n-E2B-it-litert-lm, the model name and a --prompt; try a larger model like gemma-4-E4B-it with --backend=gpu and --speculative-decoding=on if your hardware supports it. All steps.

Who it is forA developer curious whether an on-device model is good enough for their app before committing to it.

The problemTesting on-device models usually means setting up a mobile project before you see a single answer.

The outcomeA local model answering prompts in your terminal, downloaded straight from Hugging Face.

What you need first

  • uv to install the CLI with uv tool install
  • Linux, macOS, Windows or Raspberry Pi for the CLI; Android, iOS or the web through the language APIs
  • A LiteRT-LM model, for example from the litert-community Hugging Face repos

Steps

  1. Install the CLI with uv tool install litert-lm
  2. Run litert-lm run with --from-huggingface-repo pointing at a model repo such as google/gemma-3n-E2B-it-litert-lm, the model name and a --prompt
  3. Try a larger model like gemma-4-E4B-it with --backend=gpu and --speculative-decoding=on if your hardware supports it
  4. Use litert-lm describe to inspect a model's metadata, capabilities and runtime requirements before loading it
  5. Read the LiteRT LM CLI Guide for advanced usage

Prompt for your agent

Install google-ai-edge/LiteRT-LM with uv, run the gemma-3n-E2B-it-int4 model from the README with a test prompt, then try the Gemma 4 E4B example with the GPU backend. Report speed and whether each worked on my machine.

Caveat. Model downloads are large, and GPU or NPU acceleration depends on your hardware and drivers.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for google-ai-edge/LiteRT-LM and "Run Gemma on your own machine from the terminal"

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer