How to use LiteRT LM: what you can do with it

3 use cases

• Run Gemma on your own machine from the terminal

  Install the LiteRT LM CLI with uv and chat with a Gemma model without writing any code.

  • REPO google-ai-edge/LiteRT-LM
  • Claude Code
  • Codex
  • Cursor
  • Software and DevOps
  • Beginner
  View prompt

  Install google-ai-edge/LiteRT-LM with uv, run the gemma-3n-E2B-it-int4 model from the README with a test prompt, then try the Gemma 4 E4B example with the GPU backend. Report speed and whether each worked on my machine.

  Share Email me this

• Add an offline language model to an Android app

  Use the stable Kotlin API to run Gemma inside your Android app with no server.

  • REPO google-ai-edge/LiteRT-LM
  • Claude Code
  • Codex
  • Cursor
  • Software and DevOps
  • Advanced
  View prompt

  Help me add google-ai-edge/LiteRT-LM to my Android app <project> using the Kotlin API. Load <model>, add a summarize(text) function that runs on device, and show how to fall back to CPU when no GPU or NPU is available.

  Share Email me this

• Serve local embeddings through an OpenAI-compatible endpoint

  Run litert-lm serve with an embedding model to get a local /v1/embeddings endpoint for search and RAG.

  • REPO google-ai-edge/LiteRT-LM
  • Claude Code
  • Codex
  • Cursor
  • Software and DevOps
  • Intermediate
  View prompt

  Set up google-ai-edge/LiteRT-LM to serve EmbeddingGemma 2 locally with litert-lm serve, then change my <search code> to call the local /v1/embeddings endpoint instead of the hosted API. Embed <sample docs> and show a test query.

  Share Email me this

Quick start with LiteRT LM

1. Check you have what it needs: uv to install the CLI with uv tool install; Linux, macOS, Windows or Raspberry Pi for the CLI; Android, iOS or the web through the language APIs; A LiteRT-LM model, for example from the litert-community Hugging Face repos.
2. Open google-ai-edge/LiteRT-LM on GitHub and read the README and license.
3. Install the CLI with uv tool install litert-lm
4. Run litert-lm run with --from-huggingface-repo pointing at a model repo such as google/gemma-3n-E2B-it-litert-lm, the model name and a --prompt
5. Try a larger model like gemma-4-E4B-it with --backend=gpu and --speculative-decoding=on if your hardware supports it
6. Use litert-lm describe to inspect a model's metadata, capabilities and runtime requirements before loading it

These steps come from our top use case, Run Gemma on your own machine from the terminal, which also has a copy-paste agent prompt.

Using the Deduped MCP, get the setup steps for google-ai-edge/LiteRT-LM

claude mcp add --transport http deduped https://dedupedai.com/mcp

Best AI repos for Software and DevOps

About this repo

• Who it is forApp and edge developers who want a language model to run on the user's device instead of a cloud API.
• Why it is hot6.8k stars, talked about in 2 AI sources we track
• LanguageC++
• Sourcegithub.com/google-ai-edge/LiteRT-LM · Back to Hot repos

Requirements

• uv to install the CLI with uv tool install
• Linux, macOS, Windows or Raspberry Pi for the CLI; Android, iOS or the web through the language APIs
• A LiteRT-LM model, for example from the litert-community Hugging Face repos

Related repos

• Trended earlier REPO simonw/llm LLM LLM is a CLI tool and Python library for interacting with OpenAI, Gemini, Qwen, DeepSeek, Mistral and dozens of other large language models, through remote APIs or models installed on your own machine. It can run prompts from the command line, store prompts and responses in SQLite, generate embeddings, extract structured content, and let models execute tools.
• Trended earlier REPO BerriAI/litellm Litellm LiteLLM is an open source AI Gateway and Python SDK that lets you call 100+ LLM providers through one OpenAI-format interface, with cost tracking, guardrails, and load balancing.
• Trended earlier REPO langfuse/langfuse Langfuse Langfuse is an open source LLM engineering platform for developing, monitoring, evaluating and debugging AI applications. It traces LLM calls, retrieval and agent steps, manages and versions prompts, runs evaluations with LLM-as-a-judge, code evaluators and user feedback, keeps datasets for testing, and offers a playground and a full API. It can be self-hosted with Docker Compose in minutes or used as a managed cloud.