# AI use cases for Peft (huggingface/peft)

Repo: [huggingface/peft](https://github.com/huggingface/peft) | Use cases: 3 | Page: https://dedupedai.com/use-cases/huggingface-peft/

PEFT is Hugging Face's library of parameter-efficient fine-tuning methods such as LoRA. Instead of training all of a model's weights, it trains a small adapter, often well under 1% of parameters, which cuts GPU memory and storage while keeping performance close to full fine-tuning. It integrates with Transformers, Diffusers, Accelerate and TRL, and combines with quantization for training on consumer GPUs.

This file holds every use case of this repo. Each one starts with a "## " heading and ends with its URL.

- Fine-tune a small LLM with LoRA on one GPU: https://dedupedai.com/use-cases/huggingface-peft/fine-tune-a-small-llm-with-lora/
- Serve several task adapters from one base model: https://dedupedai.com/use-cases/huggingface-peft/swap-task-adapters-on-one-base-model/
- Train a custom Stable Diffusion style with LoRA: https://dedupedai.com/use-cases/huggingface-peft/train-a-custom-stable-diffusion-style-with-lora/

## How to fine-tune a small LLM with LoRA on one GPU with Peft

Repo: [huggingface/peft](https://github.com/huggingface/peft) | Difficulty: Intermediate | Time: 2 hours | Basis: From the README

Wrap a base model with a LoRA config so you train only a tiny fraction of its parameters.

**Who it is for:** A developer who wants a model that speaks in their product's style without renting a GPU cluster.

**The problem:** Full fine-tuning of even a 3B model needs a lot of GPU memory and produces huge checkpoints.

**The outcome:** A small LoRA adapter saved to disk, trained on your data, with the base model untouched.

### Steps

1. Install with `pip install peft`
2. Load a base model such as Qwen/Qwen2.5-3B-Instruct with Transformers
3. Create a `LoraConfig` with r=16 and lora_alpha=32 for a causal LM and wrap the model with `get_peft_model`
4. Call `model.print_trainable_parameters()` to confirm only about 0.12% of parameters are trainable
5. Train with the Transformers Trainer, then save with `model.save_pretrained("qwen2.5-3b-lora")`

### Prompt to paste into your agent

```text
Using huggingface/peft, fine-tune Qwen/Qwen2.5-3B-Instruct with LoRA (r=16, lora_alpha=32) on <dataset> using the Transformers Trainer. Print the trainable parameter count first, save the adapter, and report training loss.
```

### Caveat

Training time and memory depend on your GPU, sequence length and dataset size.

Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/

URL: https://dedupedai.com/use-cases/huggingface-peft/fine-tune-a-small-llm-with-lora/

## How to serve several task adapters from one base model with Peft

Repo: [huggingface/peft](https://github.com/huggingface/peft) | Difficulty: Intermediate | Time: 45 minutes | Basis: From the README

Save one small adapter per task and load or switch between them on a single base model.

**Who it is for:** A team with separate fine-tunes for support replies, summaries and tagging who cannot host three full models.

**The problem:** Keeping a full copy of a model for every task wastes storage and GPU memory.

**The outcome:** One base model in memory with small adapters you load and switch per request.

### Steps

1. Load a trained adapter for inference with `PeftModel.from_pretrained` on top of the base model
2. With a Transformers model, add adapters with `add_adapter` and load trained ones with `load_adapter`
3. Switch between them with `set_adapter`, for example to lora_2
4. Keep each adapter as its own small checkpoint per task or dataset

### Prompt to paste into your agent

```text
Using huggingface/peft, load base model <model> once and attach adapters <adapter A> and <adapter B> with load_adapter. Write a function that switches adapters with set_adapter per request and test each on <prompt>.
```

### Caveat

The Transformers integration does not include every PEFT feature, such as merging the adapter into the base model.

Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/

URL: https://dedupedai.com/use-cases/huggingface-peft/swap-task-adapters-on-one-base-model/

## How to train a custom Stable Diffusion style with LoRA with Peft

Repo: [huggingface/peft](https://github.com/huggingface/peft) | Difficulty: Advanced | Time: 2 hours | Basis: From the README

Use PEFT LoRA with Diffusers to teach an image model a subject or style on a modest GPU.

**Who it is for:** An artist or designer who wants a model that reproduces their style or product consistently.

**The problem:** Full fine-tuning of Stable Diffusion needs about 27.5GB of GPU memory and produces a large checkpoint.

**The outcome:** A small LoRA checkpoint for Stable Diffusion trained with as little as about 8GB of GPU memory with gradient checkpointing.

### Steps

1. Install PEFT with `pip install peft` and Diffusers
2. Open the examples/lora_dreambooth/train_dreambooth.py training script from the repo
3. Prepare a small set of images of your subject or style
4. Train with LoRA and gradient checkpointing to keep memory low
5. Load the resulting adapter in Diffusers to generate images

### Prompt to paste into your agent

```text
Using huggingface/peft, run examples/lora_dreambooth/train_dreambooth.py on CompVis/stable-diffusion-v1-4 with my images in <folder>, using LoRA and gradient checkpointing. Report peak GPU memory and save the adapter.
```

### Caveat

The memory numbers in the README are for CompVis/stable-diffusion-v1-4 on an A100; your results will vary.

Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/

URL: https://dedupedai.com/use-cases/huggingface-peft/train-a-custom-stable-diffusion-style-with-lora/