DEDUPED Hot AI repos. Real use cases. What people build. Get the Monday 5

How to serve Qwen3-8B with an Ada Flash drafter and test it over HTTP with Ada Flash

Launch an Ada Flash SGLang server for Qwen3-8B and send requests to its /generate endpoint.

Short answer: Complete the environment setup with conda create -n adaflash python=3.12 -y, conda activate adaflash, the patched SGLang install, and pip install -r requirements.txt; then launch the Ada Flash server with scripts/serve/serve_thresh_head.sh, which the README lists as the Ada Flash serving launcher; wait until the server reports ready, then run python bin/benchmark.py against its /generate endpoint. All steps.

Who it is forA developer who runs a Qwen3-8B inference endpoint and wants lower latency from speculative decoding without changing the target model.

The problemPlain autoregressive decoding is slow for an internal Qwen3-8B service, and the team needs a tested way to swap in a faster drafter.

The outcomeA running Ada Flash server for Qwen3-8B that answers requests on the /generate endpoint, with timings from the included benchmark client.

What you need first

  • A Linux machine with NVIDIA GPUs and CUDA, since the README sets CUDA devices and separate GPU sets for serving and training
  • Conda and Python 3.12, as used in the README install steps
  • Access to the Qwen3-8B target model and the AdaFlash/DFlash checkpoints on Hugging Face
  • The patched SGLang version from the linked SGLang PR, because a stock SGLang release may not expose the required arguments

Steps

  1. Complete the environment setup with conda create -n adaflash python=3.12 -y, conda activate adaflash, the patched SGLang install, and pip install -r requirements.txt.
  2. Launch the Ada Flash server with scripts/serve/serve_thresh_head.sh, which the README lists as the Ada Flash serving launcher.
  3. Wait until the server reports ready, then run python bin/benchmark.py against its /generate endpoint.
  4. Compare the output with the Vanilla launcher scripts/serve/serve_sglang.sh using the same benchmark client and settings.

Prompt for your agent

In the ZinYY/AdaFlash repo, start the AdaFlash server with scripts/serve/serve_thresh_head.sh for Qwen/Qwen3-8B and then run python bin/benchmark.py against its /generate endpoint. Expected output is a benchmark summary from the client. Verify that the server is ready before sending requests and that the drafter is AdaFlash/Qwen3-8B-PerfectBlend.

Caveat. The README does not document the serve script arguments, so check the script header or docs before changing its defaults for your model or GPUs.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for ZinYY/AdaFlash and "Serve Qwen3-8B with an Ada Flash drafter and test it over HTTP"

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer