DEDUPED Hot AI repos. Real use cases. What people build. Get the Monday 5

How to stream replies without freezing the page with Web LLM

Move inference into a Web Worker with WebWorkerMLCEngineHandler and stream tokens back with stream true.

Short answer: Install with npm install @mlc-ai/web-llm; then in worker.ts create a WebWorkerMLCEngineHandler and forward self.onmessage to it; in main.ts create the engine with CreateWebWorkerMLCEngine and a new Worker for worker.ts. All steps.

Who it is forA front end developer whose UI stutters while the model runs.

The problemRunning a model on the main thread blocks the interface.

The outcomeA responsive page that shows tokens as they arrive.

What you need first

  • A browser with WebGPU support on a device with a capable GPU
  • Node with npm, yarn or pnpm: `npm install @mlc-ai/web-llm`, or import from the jsdelivr CDN
  • Bandwidth and storage for the first model download, which the README warns can take a significant amount of time

Steps

  1. Install with npm install @mlc-ai/web-llm
  2. In worker.ts create a WebWorkerMLCEngineHandler and forward self.onmessage to it
  3. In main.ts create the engine with CreateWebWorkerMLCEngine and a new Worker for worker.ts
  4. Call chat.completions.create with stream: true and stream_options include_usage, appending each chunk's delta content

Prompt for your agent

With mlc-ai/web-llm, refactor <my chat page> to use CreateWebWorkerMLCEngine with a worker.ts handler and stream tokens into the message box.

Caveat. Only the last streamed chunk carries usage numbers.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for mlc-ai/web-llm and "Stream replies without freezing the page"

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer