DEDUPED Hot AI repos. Real use cases. What people build. Get the Monday 5

How to add a private chatbot that runs in the browser with Web LLM

Load a model with CreateMLCEngine and call engine.chat.completions.create with OpenAI style messages, all on the visitor's device.

Short answer: Install with npm install @mlc-ai/web-llm; then import CreateMLCEngine and create an engine for Llama-3.1-8B-Instruct-q4f32_1-MLC with an initProgressCallback to show loading progress; call engine.chat.completions.create with your messages. All steps.

  • Top pick
  • Software and DevOps
  • Intermediate
  • 1 hour
  • From the README
  • Claude Code
  • Codex
  • Cursor
  • Gemini CLI

platform. WebGPU is needed, and low end phones or laptops may be too slow or run out of memory for 8B models.

Who it is forA web developer adding an assistant to a site or internal tool.

The problemHosted LLM APIs cost money per token and send user text to a server.

The outcomeA working chat reply generated locally in the browser.

What you need first

  • A browser with WebGPU support on a device with a capable GPU
  • Node with npm, yarn or pnpm: `npm install @mlc-ai/web-llm`, or import from the jsdelivr CDN
  • Bandwidth and storage for the first model download, which the README warns can take a significant amount of time

Steps

  1. Install with npm install @mlc-ai/web-llm
  2. Import CreateMLCEngine and create an engine for Llama-3.1-8B-Instruct-q4f32_1-MLC with an initProgressCallback to show loading progress
  3. Call engine.chat.completions.create with your messages
  4. Read reply.choices[0].message and reply.usage

Prompt for your agent

Using mlc-ai/web-llm, build a minimal page that loads Llama-3.1-8B-Instruct-q4f32_1-MLC with a progress bar and answers questions with engine.chat.completions.create.

Caveat. The README says the model parameter in chat.completions.create is ignored; choose the model in CreateMLCEngine or engine.reload.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for mlc-ai/web-llm and "Add a private chatbot that runs in the browser"

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer