DEDUPED Hot AI repos, turned into use cases Get 5 a day

What you can do with STEP Quant

STEPQuant is Python code that compresses the persistent recurrent states of Delta-rule models by deciding which state units get more bits and how keys and values are scaled, with SGLang serving support.

Trended earlier

Dreamer-Toby/STEPQuant on GitHub

Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk.

What do you want to do?

3 use cases

  • Reproduce STEP Quant@6 results on Qwen3.8-27B

    Calibrate a 6-bit STEP Quant plan, serve it with SGLang, and run the long-generation evaluation suite.

    View prompt
    In the STEPQuant repo at <repo_path>, follow the Quick Start in README.md to calibrate the stepquant6 plan for the Qwen checkpoint at <model_path>, then run the evaluation suite with the fp32 and stepquant6 formats. Report the average accuracy for each format and confirm the STEPQuant@6 score is close to the FP32 score in the README results table.

  • Serve a Delta-rule model with compressed recurrent state

    Run a Delta-rule model under SGLang with STEP Quant state compression to cut per-request state memory.

    View prompt
    Using the STEPQuant repo at <repo_path>, calibrate a stepquant6 plan for the model at <model_path> and start the SGLang server with --state-format stepquant6. Report the server URL and the text of one test completion, and verify the server log shows the stepquant6 state format was loaded.

  • Compare FP32 and quantized state accuracy on long tasks

    Run the evaluation suite to score FP32 against STEP Quant@6 on long-generation benchmarks.

    View prompt
    In the STEPQuant repo at <repo_path>, run the evaluation suite on the long benchmarks for the fp32 and stepquant6 formats of the Qwen model. Report the average accuracy for each format and state whether the stepquant6 score is within 1 point of the fp32 score.

How to use STEP Quant

  1. Check you have what it needs: Model checkpoints for a supported model, such as Qwen3.8-27B, placed on disk; A CUDA toolkit and NVIDIA GPUs (the Qwen example uses four GPUs; the paper used NVIDIA A800 GPUs); Python with the tested stack: SGLang 0.5.12, PyTorch 2.11.0, Triton 3.7.1.
  2. Open Dreamer-Toby/STEPQuant on GitHub and read the README and license.
  3. Set model_path in configs/models.json to your Qwen checkpoint directory.
  4. Run bash scripts/reuse_sglang_env.sh /path/to/sglang-env, then bash scripts/reuse_eval_env.sh .venv-serving, then bash scripts/reuse_calibration_env.sh /path/to/qwen-env qwen.
  5. Run .venv-eval/bin/python -m stepquant.reproduction --models qwen --formats stepquant6 --devices 0,1,2,3 to calibrate once and write the plan. (inferred, check the README)
  6. Run CUDA_VISIBLE_DEVICES=0,1,2,3 .venv-serving/bin/python -m stepquant.sglang --model-path /path/to/Qwen3.8-27B --tp-size 4 --state-format stepquant6 --plan artifacts/reproduction/plans/qwen-stepquant6.pt --attention-backend triton --reasoning-parser qwen3 --context-length 131072 --chunked-prefill-size 2048 --mem-fraction-static 0.85 --served-model-name stepquant --host 127.0.0.1 --port 31080 to serve. (inferred, check the README)

These steps come from our top use case, Reproduce STEP Quant@6 results on Qwen3.8-27B, which also has a copy-paste agent prompt.

STEP Quant alternatives · Best AI repos for Research

About this repo

  • Who it is forWritten for ML researchers and inference engineers who run Delta-rule language models on NVIDIA GPUs and want to reproduce or serve the paper's quantized recurrent state.
  • Why it is hot
  • LanguagePython
  • Sourcegithub.com/Dreamer-Toby/STEPQuant · Back to Hot repos

Requirements

  • Model checkpoints for a supported model, such as Qwen3.8-27B, placed on disk
  • A CUDA toolkit and NVIDIA GPUs (the Qwen example uses four GPUs; the paper used NVIDIA A800 GPUs)
  • Python with the tested stack: SGLang 0.5.12, PyTorch 2.11.0, Triton 3.7.1
  • A clone of the repository, run from its root
Maintain STEP Quant? Add the Featured on Deduped badge

Paste this into your README:

[![Featured on Deduped](https://dedupedai.com/badge/dreamer-toby-stepquant.svg)](https://dedupedai.com/use-cases/dreamer-toby-stepquant/?ref=badge)

More badge styles and HTML

Questions about STEP Quant

What is STEP Quant?

STEPQuant is Python code that compresses the persistent recurrent states of Delta-rule models by deciding which state units get more bits and how keys and values are scaled, with SGLang serving support. It is an open-source project on GitHub: https://github.com/Dreamer-Toby/STEPQuant.

Who is STEP Quant for?

Written for ML researchers and inference engineers who run Delta-rule language models on NVIDIA GPUs and want to reproduce or serve the paper's quantized recurrent state.

What can I do with STEP Quant?

3 use cases so far, for example: Reproduce STEP Quant@6 results on Qwen3.8-27B; Serve a Delta-rule model with compressed recurrent state; Compare FP32 and quantized state accuracy on long tasks. Each has steps and a copy-paste agent prompt.

What do I need to run STEP Quant?

Model checkpoints for a supported model, such as Qwen3.8-27B, placed on disk; A CUDA toolkit and NVIDIA GPUs (the Qwen example uses four GPUs; the paper used NVIDIA A800 GPUs); Python with the tested stack: SGLang 0.5.12, PyTorch 2.11.0, Triton 3.7.1; A clone of the repository, run from its root.

What are alternatives to STEP Quant?

Other hot AI repos in Research: Scientific Agent Skills, Speaker Mem R1, DSH Council, Evo Bot, AI Engineering Course.

Is STEP Quant safe to run?

Deduped does not own, audit or vouch for it. Read the README and license, run it on your own systems first and review every command before you run it.