# AI use cases for STEP Quant (Dreamer-Toby/STEPQuant)

Repo: [Dreamer-Toby/STEPQuant](https://github.com/Dreamer-Toby/STEPQuant) | Use cases: 3 | Page: https://dedupedai.com/use-cases/dreamer-toby-stepquant/

STEPQuant is Python code that compresses the persistent recurrent states of Delta-rule models by deciding which state units get more bits and how keys and values are scaled, with SGLang serving support.

This file holds every use case of this repo. Each one starts with a "## " heading and ends with its URL.

- Reproduce STEP Quant@6 results on Qwen3.8-27B: https://dedupedai.com/use-cases/dreamer-toby-stepquant/reproduce-stepquant6-qwen-results/
- Serve a Delta-rule model with compressed recurrent state: https://dedupedai.com/use-cases/dreamer-toby-stepquant/serve-quantized-delta-rule-model/
- Compare FP32 and quantized state accuracy on long tasks: https://dedupedai.com/use-cases/dreamer-toby-stepquant/compare-quantized-state-accuracy/

## How to reproduce STEP Quant@6 results on Qwen3.8-27B with STEP Quant

Repo: [Dreamer-Toby/STEPQuant](https://github.com/Dreamer-Toby/STEPQuant) | Difficulty: Advanced | Time: 1 day | Basis: From the README

Calibrate a 6-bit STEP Quant plan, serve it with SGLang, and run the long-generation evaluation suite.

**Who it is for:** A researcher checking the paper's claims who has Qwen checkpoints and four GPUs available.

**The problem:** Paper numbers are hard to trust without rerunning the calibration, serving, and evaluation pipeline on the same model.

**The outcome:** A calibrated plan file, a running quantized server, and evaluation scores for FP32 and STEP Quant@6 on long-generation tasks.

### Steps

1. Set `model_path` in configs/models.json to your Qwen checkpoint directory.
2. Run `bash scripts/reuse_sglang_env.sh /path/to/sglang-env`, then `bash scripts/reuse_eval_env.sh .venv-serving`, then `bash scripts/reuse_calibration_env.sh /path/to/qwen-env qwen`.
3. Run `.venv-eval/bin/python -m stepquant.reproduction --models qwen --formats stepquant6 --devices 0,1,2,3` to calibrate once and write the plan. (inferred, check the README)
4. Run `CUDA_VISIBLE_DEVICES=0,1,2,3 .venv-serving/bin/python -m stepquant.sglang --model-path /path/to/Qwen3.8-27B --tp-size 4 --state-format stepquant6 --plan artifacts/reproduction/plans/qwen-stepquant6.pt --attention-backend triton --reasoning-parser qwen3 --context-length 131072 --chunked-prefill-size 2048 --mem-fraction-static 0.85 --served-model-name stepquant --host 127.0.0.1 --port 31080` to serve. (inferred, check the README)
5. Stop the manual server, then run `.venv-eval/bin/python -m stepquant.evaluation.suite --models qwen --benchmarks long --formats fp32 stepquant6 --server-python .venv-serving/bin/python --devices 0,1,2,3 --tp 4` and compare the scores to the paper table. (inferred, check the README)

### Prompt to paste into your agent

```text
In the STEPQuant repo at <repo_path>, follow the Quick Start in README.md to calibrate the stepquant6 plan for the Qwen checkpoint at <model_path>, then run the evaluation suite with the fp32 and stepquant6 formats. Report the average accuracy for each format and confirm the STEPQuant@6 score is close to the FP32 score in the README results table.
```

### Caveat

The README requires model checkpoints, a CUDA toolkit, and four GPUs for the Qwen example, and the paper's numbers come from NVIDIA A800 GPUs, so your scores may differ.

Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/

URL: https://dedupedai.com/use-cases/dreamer-toby-stepquant/reproduce-stepquant6-qwen-results/

## How to serve a Delta-rule model with compressed recurrent state with STEP Quant

Repo: [Dreamer-Toby/STEPQuant](https://github.com/Dreamer-Toby/STEPQuant) | Difficulty: Advanced | Time: half day | Basis: From the README

Run a Delta-rule model under SGLang with STEP Quant state compression to cut per-request state memory.

**Who it is for:** An inference engineer serving a Delta-rule model under concurrent requests, where fixed-size state still adds up in memory.

**The problem:** Recurrent state memory grows with concurrency, and uniform low-bit quantization adds errors to every state update.

**The outcome:** An OpenAI-style local endpoint, served from SGLang, that stores recurrent state in the STEP Quant 6-bit format.

### Steps

1. Run `bash scripts/reuse_sglang_env.sh /path/to/sglang-env` and `bash scripts/reuse_calibration_env.sh /path/to/qwen-env qwen` to prepare the environments.
2. Run `.venv-eval/bin/python -m stepquant.reproduction --models qwen --formats stepquant6 --devices 0,1,2,3` to produce the calibrated plan at artifacts/reproduction/plans/qwen-stepquant6.pt. (inferred, check the README)
3. Run `CUDA_VISIBLE_DEVICES=0,1,2,3 .venv-serving/bin/python -m stepquant.sglang --model-path /path/to/Qwen3.8-27B --tp-size 4 --state-format stepquant6 --plan artifacts/reproduction/plans/qwen-stepquant6.pt --attention-backend triton --served-model-name stepquant --host 127.0.0.1 --port 31080`. (inferred, check the README)
4. Send a test request to the server on 127.0.0.1 port 31080 and confirm it returns a completion.

### Prompt to paste into your agent

```text
Using the STEPQuant repo at <repo_path>, calibrate a stepquant6 plan for the model at <model_path> and start the SGLang server with --state-format stepquant6. Report the server URL and the text of one test completion, and verify the server log shows the stepquant6 state format was loaded.
```

### Caveat

The README's serving stack is pinned to SGLang 0.5.12, PyTorch 2.11.0, and Triton 3.7.1, and the example needs four GPUs, so other setups are not covered by the README.

Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/

URL: https://dedupedai.com/use-cases/dreamer-toby-stepquant/serve-quantized-delta-rule-model/

## How to compare FP32 and quantized state accuracy on long tasks with STEP Quant

Repo: [Dreamer-Toby/STEPQuant](https://github.com/Dreamer-Toby/STEPQuant) | Difficulty: Intermediate | Time: half day | Basis: From the README

Run the evaluation suite to score FP32 against STEP Quant@6 on long-generation benchmarks.

**Who it is for:** A model evaluator who needs accuracy evidence before deciding whether a quantized recurrent state is safe to deploy.

**The problem:** Memory savings mean little if long-generation accuracy drops, and the comparison has to be run on the same model.

**The outcome:** Average accuracy scores for FP32 and STEP Quant@6 on the long-generation benchmark set, to compare against the paper table.

### Steps

1. Complete the calibration step so the stepquant6 plan exists at artifacts/reproduction/plans/qwen-stepquant6.pt.
2. Stop any manually started server, since the evaluation suite starts its own.
3. Run `.venv-eval/bin/python -m stepquant.evaluation.suite --models qwen --benchmarks long --formats fp32 stepquant6 --server-python .venv-serving/bin/python --devices 0,1,2,3 --tp 4`. (inferred, check the README)
4. Open REPRODUCTION.md for the zero-shot short-task protocol if you also want short-task scores. (inferred, check the README)

### Prompt to paste into your agent

```text
In the STEPQuant repo at <repo_path>, run the evaluation suite on the long benchmarks for the fp32 and stepquant6 formats of the Qwen model. Report the average accuracy for each format and state whether the stepquant6 score is within 1 point of the fp32 score.
```

### Caveat

The README notes that the 4 and 6 bit budgets are nominal because FP16 pivots and scales add storage, so reported accuracy should be read with that in mind.

Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/

URL: https://dedupedai.com/use-cases/dreamer-toby-stepquant/compare-quantized-state-accuracy/