DEDUPED Hot AI repos. Real use cases. What people build. Get the Monday 5

How to use Web LLM: what you can do with it

What is Web LLM? WebLLM is an in-browser LLM inference engine that runs models such as Llama 3, Phi 3, Gemma, Mistral and Qwen entirely on the user's device with WebGPU, with no server. It exposes an OpenAI compatible chat completions API with streaming and JSON mode, and supports Web Workers, Service Workers and Chrome extensions.

How do I use it? Install with npm install @mlc-ai/web-llm; then import CreateMLCEngine and create an engine for Llama-3.1-8B-Instruct-q4f32_1-MLC with an initProgressCallback to show loading progress; call engine.chat.completions.create with your messages. Full steps below.

Trended earlier

View repo on GitHub

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer

3 use cases

Quick start with Web LLM

  1. Check you have what it needs: A browser with WebGPU support on a device with a capable GPU; Node with npm, yarn or pnpm: `npm install @mlc-ai/web-llm`, or import from the jsdelivr CDN; Bandwidth and storage for the first model download, which the README warns can take a significant amount of time.
  2. Open mlc-ai/web-llm on GitHub and read the README and license.
  3. Install with npm install @mlc-ai/web-llm
  4. Import CreateMLCEngine and create an engine for Llama-3.1-8B-Instruct-q4f32_1-MLC with an initProgressCallback to show loading progress
  5. Call engine.chat.completions.create with your messages
  6. Read reply.choices[0].message and reply.usage

These steps come from our top use case, Add a private chatbot that runs in the browser, which also has a copy-paste agent prompt.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for mlc-ai/web-llm

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Best AI repos for Software and DevOps

About this repo

  • Who it is forWeb developers who want private, offline capable AI features without paying for inference servers.
  • Why it is hot19k stars, talked about in 2 AI sources we track
  • Sourcegithub.com/mlc-ai/web-llm ยท Back to Hot repos

Requirements

  • A browser with WebGPU support on a device with a capable GPU
  • Node with npm, yarn or pnpm: `npm install @mlc-ai/web-llm`, or import from the jsdelivr CDN
  • Bandwidth and storage for the first model download, which the README warns can take a significant amount of time

Add this badge to your README

Maintain Web LLM? Show readers where to find these use cases. Use cases on Deduped

Markdown

[![Use cases on Deduped](https://dedupedai.com/badge/mlc-ai/web-llm.svg)](https://dedupedai.com/use-cases/mlc-ai-web-llm/?utm_source=github&utm_medium=badge&utm_campaign=mlc-ai-web-llm)

HTML

<a href="https://dedupedai.com/use-cases/mlc-ai-web-llm/?utm_source=github&amp;utm_medium=badge&amp;utm_campaign=mlc-ai-web-llm"><img src="https://dedupedai.com/badge/mlc-ai/web-llm.svg" alt="Use cases on Deduped" height="20"></a>

A static image, no tracking script. About the badge

Questions about Web LLM

What is Web LLM?

WebLLM is an in-browser LLM inference engine that runs models such as Llama 3, Phi 3, Gemma, Mistral and Qwen entirely on the user's device with WebGPU, with no server. It exposes an OpenAI compatible chat completions API with streaming and JSON mode, and supports Web Workers, Service Workers and Chrome extensions. It is an open-source project on GitHub: https://github.com/mlc-ai/web-llm.

How do I use Web LLM?

Install with npm install @mlc-ai/web-llm; then import CreateMLCEngine and create an engine for Llama-3.1-8B-Instruct-q4f32_1-MLC with an initProgressCallback to show loading progress; call engine.chat.completions.create with your messages. The use cases on this page give the full steps and a copy-paste agent prompt for each job.

Who is Web LLM for?

Web developers who want private, offline capable AI features without paying for inference servers.

What can I do with Web LLM?

3 use cases so far, for example: Add a private chatbot that runs in the browser; Stream replies without freezing the page; Keep the model loaded across page visits. Each has steps and a copy-paste agent prompt.

What do I need to run Web LLM?

A browser with WebGPU support on a device with a capable GPU; Node with npm, yarn or pnpm: `npm install @mlc-ai/web-llm`, or import from the jsdelivr CDN; Bandwidth and storage for the first model download, which the README warns can take a significant amount of time.

What are alternatives to Web LLM?

Other hot AI repos in Software and DevOps: Skills, Claude Mem, Freellmapi, Andrej Karpathy Skills, Agent Desktop.

Is Web LLM safe to run?

Deduped does not own, audit or vouch for it. Read the README and license, run it on your own systems first and review every command before you run it.