What do you want to do?
3 use cases
-
Add a private chatbot that runs in the browser
Load a model with CreateMLCEngine and call engine.chat.completions.create with OpenAI style messages, all on the visitor's device.
View prompt
Using mlc-ai/web-llm, build a minimal page that loads Llama-3.1-8B-Instruct-q4f32_1-MLC with a progress bar and answers questions with engine.chat.completions.create.
-
Stream replies without freezing the page
Move inference into a Web Worker with WebWorkerMLCEngineHandler and stream tokens back with stream true.
View prompt
With mlc-ai/web-llm, refactor <my chat page> to use CreateWebWorkerMLCEngine with a worker.ts handler and stream tokens into the message box.
-
Keep the model loaded across page visits
Run WebLLM in a Service Worker and pick a cache backend so the model is not reloaded on every visit and works better offline.
View prompt
Using mlc-ai/web-llm, add a sw.ts with ServiceWorkerMLCEngineHandler, register it, create the engine with CreateServiceWorkerMLCEngine, and set cacheBackend to indexeddb.
Quick start with Web LLM
- Check you have what it needs: A browser with WebGPU support on a device with a capable GPU; Node with npm, yarn or pnpm: `npm install @mlc-ai/web-llm`, or import from the jsdelivr CDN; Bandwidth and storage for the first model download, which the README warns can take a significant amount of time.
- Open mlc-ai/web-llm on GitHub and read the README and license.
- Install with
npm install @mlc-ai/web-llm - Import CreateMLCEngine and create an engine for Llama-3.1-8B-Instruct-q4f32_1-MLC with an initProgressCallback to show loading progress
- Call engine.chat.completions.create with your messages
- Read reply.choices[0].message and reply.usage
These steps come from our top use case, Add a private chatbot that runs in the browser, which also has a copy-paste agent prompt.
Use this in your AI agent
Ask your agent
Using the Deduped MCP, get the setup steps for mlc-ai/web-llm
Add Deduped to Claude Code once
claude mcp add --transport http deduped https://dedupedai.com/mcp
Every Monday: the 5 hottest repos and exactly how to use them. Free, one email a week, unsubscribe anytime.
Get them freeAbout this repo
- Who it is forWeb developers who want private, offline capable AI features without paying for inference servers.
- Why it is hot19k stars, talked about in 2 AI sources we track
- Sourcegithub.com/mlc-ai/web-llm ยท Back to Hot repos
Requirements
- A browser with WebGPU support on a device with a capable GPU
- Node with npm, yarn or pnpm: `npm install @mlc-ai/web-llm`, or import from the jsdelivr CDN
- Bandwidth and storage for the first model download, which the README warns can take a significant amount of time
Add this badge to your README
Maintain Web LLM? Show readers where to find these use cases.
Markdown
[](https://dedupedai.com/use-cases/mlc-ai-web-llm/?utm_source=github&utm_medium=badge&utm_campaign=mlc-ai-web-llm)
HTML
<a href="https://dedupedai.com/use-cases/mlc-ai-web-llm/?utm_source=github&utm_medium=badge&utm_campaign=mlc-ai-web-llm"><img src="https://dedupedai.com/badge/mlc-ai/web-llm.svg" alt="Use cases on Deduped" height="20"></a>
A static image, no tracking script. About the badge
Questions about Web LLM
What is Web LLM?
WebLLM is an in-browser LLM inference engine that runs models such as Llama 3, Phi 3, Gemma, Mistral and Qwen entirely on the user's device with WebGPU, with no server. It exposes an OpenAI compatible chat completions API with streaming and JSON mode, and supports Web Workers, Service Workers and Chrome extensions. It is an open-source project on GitHub: https://github.com/mlc-ai/web-llm.
How do I use Web LLM?
Install with npm install @mlc-ai/web-llm; then import CreateMLCEngine and create an engine for Llama-3.1-8B-Instruct-q4f32_1-MLC with an initProgressCallback to show loading progress; call engine.chat.completions.create with your messages. The use cases on this page give the full steps and a copy-paste agent prompt for each job.
Who is Web LLM for?
Web developers who want private, offline capable AI features without paying for inference servers.
What can I do with Web LLM?
3 use cases so far, for example: Add a private chatbot that runs in the browser; Stream replies without freezing the page; Keep the model loaded across page visits. Each has steps and a copy-paste agent prompt.
What do I need to run Web LLM?
A browser with WebGPU support on a device with a capable GPU; Node with npm, yarn or pnpm: `npm install @mlc-ai/web-llm`, or import from the jsdelivr CDN; Bandwidth and storage for the first model download, which the README warns can take a significant amount of time.
What are alternatives to Web LLM?
Other hot AI repos in Software and DevOps: Skills, Claude Mem, Freellmapi, Andrej Karpathy Skills, Agent Desktop.
Is Web LLM safe to run?
Deduped does not own, audit or vouch for it. Read the README and license, run it on your own systems first and review every command before you run it.