DEDUPED Hot AI repos. Real use cases. What people build. Get the Monday 5

How to measure Ada Flash decoding speed against baselines on GSM8K with Ada Flash

Run the one-command Ada Flash benchmark on a GSM8K split to get throughput numbers for your own GPUs.

Short answer: Create the environment with conda create -n adaflash python=3.12 -y and then conda activate adaflash; then install the patched SGLang with pip install "sglang[all] @ git+https://github.com/sgl-project/sglang.git@refs/pull/34171/head#subdirectory=python" and then pip install -r requirements.txt; Prepare the benchmark split with python bin/prepare_data.py --dataset gsm8k. All steps.

Who it is forAn ML engineer deciding whether Ada Flash is worth adopting for a Qwen3-8B serving setup and needing numbers from their own hardware.

The problemPublished speedups may not match your hardware, batch sizes, or workload, so you need a repeatable local measurement before changing serving infrastructure.

The outcomeA benchmark run over 1024 GSM8K samples at concurrency 64 with the Ada Flash drafter, plus a documented protocol for comparing against other methods.

What you need first

  • A Linux machine with NVIDIA GPUs and CUDA, since the README sets CUDA devices and separate GPU sets for serving and training
  • Conda and Python 3.12, as used in the README install steps
  • Access to the Qwen3-8B target model and the AdaFlash/DFlash checkpoints on Hugging Face
  • The patched SGLang version from the linked SGLang PR, because a stock SGLang release may not expose the required arguments

Steps

  1. Create the environment with conda create -n adaflash python=3.12 -y and then conda activate adaflash.
  2. Install the patched SGLang with pip install "sglang[all] @ git+https://github.com/sgl-project/sglang.git@refs/pull/34171/head#subdirectory=python" and then pip install -r requirements.txt.
  3. Prepare the benchmark split with python bin/prepare_data.py --dataset gsm8k.
  4. Set export MODEL_PATH=Qwen/Qwen3-8B, export DRAFT_MODEL_PATH=AdaFlash/Qwen3-8B-PerfectBlend, export DATASET=gsm8k, export NUM_SAMPLES=1024, and export CONCURRENCY=64.
  5. Run bash scripts/tools/benchmark.sh and read the results it writes, then follow docs/benchmark_experiments.md to run the other baselines under the same settings.

Prompt for your agent

In the ZinYY/AdaFlash repo, set up the environment from the README, prepare the gsm8k split, and run scripts/tools/benchmark.sh with MODEL_PATH=Qwen/Qwen3-8B and DRAFT_MODEL_PATH=AdaFlash/Qwen3-8B-PerfectBlend for <your GPU machine>. Report the throughput output from the benchmark. Verify that the server started with the patched SGLang before trusting any number.

Caveat. The README says a stock SGLang release may not work, so the patched SGLang PR must be installed before the benchmark will run.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for ZinYY/AdaFlash and "Measure Ada Flash decoding speed against baselines on GSM8K"

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer