DEDUPED Hot AI repos. Real use cases. What people build. Get the Monday 5

How to train an Ada Flash drafter on your own response data with Ada Flash

Regenerate training data, initialize the adaptive length head, then run asynchronous on-policy training.

Short answer: Prepare the base dataset with python bin/prepare_data.py --dataset perfectblend, then regenerate responses with CUDA_VISIBLE_DEVICES=0 python bin/regenerate_dataset.py --model-path Qwen/Qwen3-8B --initial-draft-path z-lab/Qwen3-8B-DFlash-b16 --dataset-path cache/dataset/perfectblend_train.jsonl --num-samples 4096 --output-jsonl cache/dataset/perfectblend_qwen3_8b_regen_4096.jsonl. All steps.

Who it is forA researcher or ML engineer who wants a drafter tuned to a different dataset or target model than the released checkpoint.

The problemThe released Qwen3-8B drafter was trained on PerfectBlend, so a different domain may need its own drafter trained on matching target-model responses.

The outcomeA trained Ada Flash drafter with an initialized adaptive length head, saved as versioned weights that the async pipeline publishes for hot reloading.

What you need first

  • A Linux machine with NVIDIA GPUs and CUDA, since the README sets CUDA devices and separate GPU sets for serving and training
  • Conda and Python 3.12, as used in the README install steps
  • Access to the Qwen3-8B target model and the AdaFlash/DFlash checkpoints on Hugging Face
  • The patched SGLang version from the linked SGLang PR, because a stock SGLang release may not expose the required arguments

Steps

  1. Prepare the base dataset with python bin/prepare_data.py --dataset perfectblend, then regenerate responses with CUDA_VISIBLE_DEVICES=0 python bin/regenerate_dataset.py --model-path Qwen/Qwen3-8B --initial-draft-path z-lab/Qwen3-8B-DFlash-b16 --dataset-path cache/dataset/perfectblend_train.jsonl --num-samples 4096 --output-jsonl cache/dataset/perfectblend_qwen3_8b_regen_4096.jsonl. (inferred, check the README)
  2. Initialize the adaptive length head with the torchrun --standalone --nproc_per_node=1 --module pipeline.offline.init_thresh_head command in the README, pointing --dataset-path at your regenerated file. (inferred, check the README)
  3. Export MODEL_PATH, INITIAL_DRAFT_PATH, DRAFT_CONFIG_PATH, DATASET_PATH, SWAP_DIR, and LOG_DIR as shown in the README training section.
  4. Start training with bash scripts/pipeline/run_async_pipeline_adaflash.sh "0" "1,2", where the first argument is the inference GPUs and the second is the training GPUs, then check the logs directory.

Prompt for your agent

In the ZinYY/AdaFlash repo, prepare a dataset, regenerate responses with Qwen/Qwen3-8B, initialize the adaptive length head, and start scripts/pipeline/run_async_pipeline_adaflash.sh with inference GPU 0 and training GPUs 1,2. Expected output is versioned drafter weights written under the swap directory and logs under the log directory. Verify that the initialized draft path exists before starting the async run.

Caveat. The README gives GPU counts only as examples, so the needed GPU setup and run length for a custom dataset are not documented and must be determined by the user.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for ZinYY/AdaFlash and "Train an Ada Flash drafter on your own response data"

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer