# AI use cases for Web LLM (mlc-ai/web-llm)

Repo: [mlc-ai/web-llm](https://github.com/mlc-ai/web-llm) | Use cases: 3 | Page: https://dedupedai.com/use-cases/mlc-ai-web-llm/

WebLLM is an in-browser LLM inference engine that runs models such as Llama 3, Phi 3, Gemma, Mistral and Qwen entirely on the user's device with WebGPU, with no server. It exposes an OpenAI compatible chat completions API with streaming and JSON mode, and supports Web Workers, Service Workers and Chrome extensions.

This file holds every use case of this repo. Each one starts with a "## " heading and ends with its URL.

- Add a private chatbot that runs in the browser: https://dedupedai.com/use-cases/mlc-ai-web-llm/add-a-private-in-browser-chatbot-with-web-llm/
- Stream replies without freezing the page: https://dedupedai.com/use-cases/mlc-ai-web-llm/stream-replies-from-a-web-worker-with-web-llm/
- Keep the model loaded across page visits: https://dedupedai.com/use-cases/mlc-ai-web-llm/keep-the-model-loaded-across-pages-with-a-web-llm-service-wo/

## How to add a private chatbot that runs in the browser with Web LLM

Repo: [mlc-ai/web-llm](https://github.com/mlc-ai/web-llm) | Difficulty: Intermediate | Time: 1 hour | Basis: From the README

Load a model with CreateMLCEngine and call engine.chat.completions.create with OpenAI style messages, all on the visitor's device.

**Who it is for:** A web developer adding an assistant to a site or internal tool.

**The problem:** Hosted LLM APIs cost money per token and send user text to a server.

**The outcome:** A working chat reply generated locally in the browser.

### Steps

1. Install with `npm install @mlc-ai/web-llm`
2. Import CreateMLCEngine and create an engine for Llama-3.1-8B-Instruct-q4f32_1-MLC with an initProgressCallback to show loading progress
3. Call engine.chat.completions.create with your messages
4. Read reply.choices[0].message and reply.usage

### Prompt to paste into your agent

```text
Using mlc-ai/web-llm, build a minimal page that loads Llama-3.1-8B-Instruct-q4f32_1-MLC with a progress bar and answers questions with engine.chat.completions.create.
```

### Caveat

The README says the model parameter in chat.completions.create is ignored; choose the model in CreateMLCEngine or engine.reload.

Risk, platform: WebGPU is needed, and low end phones or laptops may be too slow or run out of memory for 8B models.

Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/

URL: https://dedupedai.com/use-cases/mlc-ai-web-llm/add-a-private-in-browser-chatbot-with-web-llm/

## How to stream replies without freezing the page with Web LLM

Repo: [mlc-ai/web-llm](https://github.com/mlc-ai/web-llm) | Difficulty: Intermediate | Time: 1 hour | Basis: From the README

Move inference into a Web Worker with WebWorkerMLCEngineHandler and stream tokens back with stream true.

**Who it is for:** A front end developer whose UI stutters while the model runs.

**The problem:** Running a model on the main thread blocks the interface.

**The outcome:** A responsive page that shows tokens as they arrive.

### Steps

1. Install with `npm install @mlc-ai/web-llm`
2. In worker.ts create a WebWorkerMLCEngineHandler and forward self.onmessage to it
3. In main.ts create the engine with CreateWebWorkerMLCEngine and a new Worker for worker.ts
4. Call chat.completions.create with stream: true and stream_options include_usage, appending each chunk's delta content

### Prompt to paste into your agent

```text
With mlc-ai/web-llm, refactor <my chat page> to use CreateWebWorkerMLCEngine with a worker.ts handler and stream tokens into the message box.
```

### Caveat

Only the last streamed chunk carries usage numbers.

Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/

URL: https://dedupedai.com/use-cases/mlc-ai-web-llm/stream-replies-from-a-web-worker-with-web-llm/

## How to keep the model loaded across page visits with Web LLM

Repo: [mlc-ai/web-llm](https://github.com/mlc-ai/web-llm) | Difficulty: Advanced | Time: 2 hours | Basis: From the README

Run WebLLM in a Service Worker and pick a cache backend so the model is not reloaded on every visit and works better offline.

**Who it is for:** A developer shipping a multi page app or PWA with local AI.

**The problem:** Reloading a multi gigabyte model on each page view is slow.

**The outcome:** A service worker engine shared across pages, with model files cached in the browser.

### Steps

1. Install with `npm install @mlc-ai/web-llm`
2. In sw.ts instantiate ServiceWorkerMLCEngineHandler at the top level of the script
3. Register the service worker in main.ts and create the engine with CreateServiceWorkerMLCEngine
4. Optionally set AppConfig.cacheBackend to cache, indexeddb or opfs

### Prompt to paste into your agent

```text
Using mlc-ai/web-llm, add a sw.ts with ServiceWorkerMLCEngineHandler, register it, create the engine with CreateServiceWorkerMLCEngine, and set cacheBackend to indexeddb.
```

### Caveat

The README warns the browser can kill a service worker at any time, so add error handling.

Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/

URL: https://dedupedai.com/use-cases/mlc-ai-web-llm/keep-the-model-loaded-across-pages-with-a-web-llm-service-wo/
