# AI use cases for LiteRT LM (google-ai-edge/LiteRT-LM)

Repo: [google-ai-edge/LiteRT-LM](https://github.com/google-ai-edge/LiteRT-LM) | Use cases: 3 | Page: https://dedupedai.com/use-cases/google-ai-edge-litert-lm/

LiteRT-LM is Google's production-ready orchestration layer for running LLMs with LiteRT on device. It runs on Android, iOS, web, desktop and IoT boards such as Raspberry Pi, uses GPU and NPU acceleration, supports vision and audio inputs and function calling, and handles Gemma, Llama, Phi-4, Qwen and more. It powers on-device AI in Chrome, Chromebook Plus and Pixel Watch, and ships a CLI plus Python, Kotlin, Swift, JavaScript and C++ APIs.

This file holds every use case of this repo. Each one starts with a "## " heading and ends with its URL.

- Run Gemma on your own machine from the terminal: https://dedupedai.com/use-cases/google-ai-edge-litert-lm/try-an-on-device-llm-from-the-terminal/
- Add an offline language model to an Android app: https://dedupedai.com/use-cases/google-ai-edge-litert-lm/add-an-offline-llm-to-an-android-app/
- Serve local embeddings through an OpenAI-compatible endpoint: https://dedupedai.com/use-cases/google-ai-edge-litert-lm/serve-local-embeddings-with-an-openai-compatible-api/

## How to run Gemma on your own machine from the terminal with LiteRT LM

Repo: [google-ai-edge/LiteRT-LM](https://github.com/google-ai-edge/LiteRT-LM) | Difficulty: Beginner | Time: 15 minutes | Basis: From the README

Install the LiteRT LM CLI with uv and chat with a Gemma model without writing any code.

**Who it is for:** A developer curious whether an on-device model is good enough for their app before committing to it.

**The problem:** Testing on-device models usually means setting up a mobile project before you see a single answer.

**The outcome:** A local model answering prompts in your terminal, downloaded straight from Hugging Face.

### Steps

1. Install the CLI with `uv tool install litert-lm`
2. Run litert-lm run with --from-huggingface-repo pointing at a model repo such as google/gemma-3n-E2B-it-litert-lm, the model name and a --prompt
3. Try a larger model like gemma-4-E4B-it with --backend=gpu and --speculative-decoding=on if your hardware supports it
4. Use `litert-lm describe` to inspect a model's metadata, capabilities and runtime requirements before loading it
5. Read the LiteRT LM CLI Guide for advanced usage

### Prompt to paste into your agent

```text
Install google-ai-edge/LiteRT-LM with uv, run the gemma-3n-E2B-it-int4 model from the README with a test prompt, then try the Gemma 4 E4B example with the GPU backend. Report speed and whether each worked on my machine.
```

### Caveat

Model downloads are large, and GPU or NPU acceleration depends on your hardware and drivers.

Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/

URL: https://dedupedai.com/use-cases/google-ai-edge-litert-lm/try-an-on-device-llm-from-the-terminal/

## How to add an offline language model to an Android app with LiteRT LM

Repo: [google-ai-edge/LiteRT-LM](https://github.com/google-ai-edge/LiteRT-LM) | Difficulty: Advanced | Time: half a day | Basis: From the README

Use the stable Kotlin API to run Gemma inside your Android app with no server.

**Who it is for:** An Android developer who wants a private, offline AI feature such as summarizing notes on the phone.

**The problem:** Cloud LLM features cost money per call, need a connection and send user data off the device.

**The outcome:** An Android app that runs a language model locally, using the GPU or NPU when available.

### Steps

1. Try models first in the Google AI Edge Gallery app to pick one that runs well on your target phones
2. Follow the Kotlin Guide, which the README marks as stable for Android apps and JVM
3. Bundle or download a LiteRT LM model such as a Gemma variant
4. Use Tool Use, the function calling support, if the feature needs to trigger actions in your app
5. Test on low-end devices too, since memory and speed vary widely

### Prompt to paste into your agent

```text
Help me add google-ai-edge/LiteRT-LM to my Android app <project> using the Kotlin API. Load <model>, add a summarize(text) function that runs on device, and show how to fall back to CPU when no GPU or NPU is available.
```

### Caveat

The README only links to the Kotlin guide, so the integration details come from Google's docs rather than this repo.

Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/

URL: https://dedupedai.com/use-cases/google-ai-edge-litert-lm/add-an-offline-llm-to-an-android-app/

## How to serve local embeddings through an OpenAI-compatible endpoint with LiteRT LM

Repo: [google-ai-edge/LiteRT-LM](https://github.com/google-ai-edge/LiteRT-LM) | Difficulty: Intermediate | Time: 45 minutes | Basis: From the README

Run litert-lm serve with an embedding model to get a local /v1/embeddings endpoint for search and RAG.

**Who it is for:** A developer building semantic search who wants embeddings without paying a hosted embeddings API.

**The problem:** Embedding a large document set through a paid API costs money and sends every document to a third party.

**The outcome:** A local endpoint your existing OpenAI client code can call for text, vision or audio embeddings.

### Steps

1. Install the CLI with `uv tool install litert-lm`
2. Import a model quickly with `litert-lm import` if you have one locally
3. Start `litert-lm serve` with a multimodal embedding model such as EmbeddingGemma 2
4. Point your OpenAI-compatible client at the local /v1/embeddings endpoint
5. Use Matryoshka dimension truncation to store shorter vectors when space matters

### Prompt to paste into your agent

```text
Set up google-ai-edge/LiteRT-LM to serve EmbeddingGemma 2 locally with litert-lm serve, then change my <search code> to call the local /v1/embeddings endpoint instead of the hosted API. Embed <sample docs> and show a test query.
```

### Caveat

The serve endpoint is described in the release notes; check the DevSite docs for the exact model names and flags in your version.

Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/

URL: https://dedupedai.com/use-cases/google-ai-edge-litert-lm/serve-local-embeddings-with-an-openai-compatible-api/