# AI use cases for Diffusion OPSD (worldbench/DiffusionOPSD)

Repo: [worldbench/DiffusionOPSD](https://github.com/worldbench/DiffusionOPSD) | Use cases: 3 | Page: https://dedupedai.com/use-cases/worldbench-diffusionopsd/

DiffusionOPSD is an external Python implementation of an on-policy self-distillation method for reward-guided post-training of diffusion models such as SD3.5-M and Z-Image-Turbo.

This file holds every use case of this repo. Each one starts with a "## " heading and ends with its URL.

- Post-train SD3.5-M toward a chosen image reward: https://dedupedai.com/use-cases/worldbench-diffusionopsd/post-train-sd35-on-image-reward/
- Verify reward models load before a long training run: https://dedupedai.com/use-cases/worldbench-diffusionopsd/check-reward-stack-before-training/
- Train the Z-Image-Turbo backbone at 1024 resolution: https://dedupedai.com/use-cases/worldbench-diffusionopsd/train-z-image-turbo-at-1024/

## How to post-train SD3.5-M toward a chosen image reward with Diffusion OPSD

Repo: [worldbench/DiffusionOPSD](https://github.com/worldbench/DiffusionOPSD) | Difficulty: Advanced | Time: 1 day | Basis: From the README

Fine-tune SD3.5-M with Diffusion OPSD using a reward preset and measure the held-out score.

**Who it is for:** An ML engineer with a multi-GPU Linux box who wants to improve SD3.5-M outputs for a specific aesthetic or preference score.

**The problem:** Reward-guided diffusion training usually sees only a final image score, so intermediate denoising steps get weak supervision.

**The outcome:** A DiffusionOPSD-trained SD3.5-M checkpoint and training logs for one reward objective, ready to compare against the README's reported held-out results.

### Steps

1. Run `git clone https://github.com/worldbench/DiffusionOPSD.git` and `cd DiffusionOPSD`.
2. Run `conda create -n diffusionopsd python=3.11 -y` and `conda activate diffusionopsd`.
3. Run `pip install -e ".[rewards]"` and then `pip install --no-deps 'image-reward==1.5'`.
4. Set `export REWARD_CKPT_PATH="$PWD/reward_ckpts"` and run `bash scripts/download_reward_weights.sh`.
5. Accept the SD3.5-M license on Hugging Face and authenticate, then start a preset with `bash scripts/train_public.sh sd35 hpsv3` (replace hpsv3 with a standard reward preset such as PickScore, which is an inferred choice).

### Prompt to paste into your agent

```text
Using the repo worldbench/DiffusionOPSD, set up the environment and run the SD3.5-M training preset for <reward name>. Report the training command used, the reward checkpoint path, and the final evaluation score for <reward name>. Verify that scripts/download_reward_weights.sh completed without errors before starting training.
```

### Caveat

The README says a recent Linux environment with CUDA GPUs is required for training, so this will not run on a laptop without suitable GPUs.

Risk, credentials: SD3.5-M is gated on Hugging Face, so you must authenticate with your own account and accept the model license before downloading.

Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/

URL: https://dedupedai.com/use-cases/worldbench-diffusionopsd/post-train-sd35-on-image-reward/

## How to verify reward models load before a long training run with Diffusion OPSD

Repo: [worldbench/DiffusionOPSD](https://github.com/worldbench/DiffusionOPSD) | Difficulty: Advanced | Time: 2 hours | Basis: From the README

Run the README setup checks and gradient smoke tests so a failed reward stack does not waste GPU hours.

**Who it is for:** A researcher about to start a costly run who needs to confirm HPSv3 or DeQA work on their machine.

**The problem:** Heavy reward models use separate dependency stacks, and a broken install can fail only after training has begun.

**The outcome:** Pass or fail results for the HPSv3 and DeQA setup checks and differentiable reward path, so the team knows the stack is usable.

### Steps

1. Create a fresh Python 3.10 or 3.11 environment and install a CUDA-matched PyTorch build first, then run `pip install -e .`.
2. Run `pip install --no-deps 'hpsv3==1.0.0'` and then the listed `pip install` line with qwen-vl-utils, omegaconf, safetensors, and the other packages from the README.
3. Run `python scripts/check_reward_setup.py --backbone sd35 --reward hpsv3`.
4. Run `python scripts/check_reward_setup.py --backbone sd35 --reward deqa`.
5. Run `python scripts/smoke_reward_gradient.py --reward hpsv3` and `python scripts/smoke_reward_gradient.py --reward deqa`.

### Prompt to paste into your agent

```text
In the repo worldbench/DiffusionOPSD, run the reward setup checks for <reward name> with the SD3.5 backbone and the gradient smoke test. Report which commands passed or failed and the error text for any failure. Verify that the smoke test ran the differentiable reward path before reporting success.
```

### Caveat

The README warns against normal dependency resolution for HPSv3 and says a separate heavy-reward environment is needed, so mixing it into the main environment can break the Transformers version.

Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/

URL: https://dedupedai.com/use-cases/worldbench-diffusionopsd/check-reward-stack-before-training/

## How to train the Z-Image-Turbo backbone at 1024 resolution with Diffusion OPSD

Repo: [worldbench/DiffusionOPSD](https://github.com/worldbench/DiffusionOPSD) | Difficulty: Advanced | Time: 1 day | Basis: From the README

Run Diffusion OPSD on the few-step Z-Image-Turbo regime at 1024 pixels.

**Who it is for:** A researcher who wants a few-step, high-resolution diffusion model tuned for a reward and has a Diffusers build with ZImagePipeline.

**The problem:** Z-Image-Turbo needs a specific Diffusers build, and the setup is different from the SD3.5-M path.

**The outcome:** A Z-Image-Turbo checkpoint trained at 1024 pixels with the Diffusion OPSD recipe, along with the training efficiency numbers to compare with DiffusionNFT.

### Steps

1. Complete the base install with `pip install -e ".[rewards]"` and `pip install --no-deps 'image-reward==1.5'`.
2. Check whether ZImagePipeline is available in your Diffusers install. If it is missing, run `git clone https://github.com/huggingface/diffusers.git` and `pip install -e "./diffusers[torch]"`.
3. Set `export REWARD_CKPT_PATH="$PWD/reward_ckpts"` and run `bash scripts/download_reward_weights.sh`.
4. Start the Z-Image-Turbo preset by passing the z-image backbone name to `bash scripts/train_public.sh` and check scripts/train_public.sh for the exact argument. (inferred, check the README)

### Prompt to paste into your agent

```text
In the repo worldbench/DiffusionOPSD, confirm ZImagePipeline imports in the installed Diffusers, then start the Z-Image-Turbo training preset for <reward name>. Report the import check result, the launch command, and the first logged seconds per update. Verify that the import check passed before launching training.
```

### Caveat

The README names the 1024 Z-Image-Turbo regime but does not give the exact launch argument for it, so the step is inferred and must be checked in the launcher script.

Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/

URL: https://dedupedai.com/use-cases/worldbench-diffusionopsd/train-z-image-turbo-at-1024/