DEDUPED Hot AI repos. Real use cases. What people build. Get the Monday 5

How to fine-tune a small LLM with LoRA on one GPU with Peft

Wrap a base model with a LoRA config so you train only a tiny fraction of its parameters.

Short answer: Install with pip install peft; then load a base model such as Qwen/Qwen2.5-3B-Instruct with Transformers; create a LoraConfig with r=16 and lora_alpha=32 for a causal LM and wrap the model with get_peft_model. All steps.

Who it is forA developer who wants a model that speaks in their product's style without renting a GPU cluster.

The problemFull fine-tuning of even a 3B model needs a lot of GPU memory and produces huge checkpoints.

The outcomeA small LoRA adapter saved to disk, trained on your data, with the base model untouched.

What you need first

  • Python with PyTorch and Transformers
  • A GPU with enough memory for the base model (the README shows a 3B model on far less memory with LoRA)
  • A dataset for your task

Steps

  1. Install with pip install peft
  2. Load a base model such as Qwen/Qwen2.5-3B-Instruct with Transformers
  3. Create a LoraConfig with r=16 and lora_alpha=32 for a causal LM and wrap the model with get_peft_model
  4. Call model.print_trainable_parameters() to confirm only about 0.12% of parameters are trainable
  5. Train with the Transformers Trainer, then save with model.save_pretrained("qwen2.5-3b-lora")

Prompt for your agent

Using huggingface/peft, fine-tune Qwen/Qwen2.5-3B-Instruct with LoRA (r=16, lora_alpha=32) on <dataset> using the Transformers Trainer. Print the trainable parameter count first, save the adapter, and report training loss.

Caveat. Training time and memory depend on your GPU, sequence length and dataset size.

Use this in your AI agent

Ask your agent

Using the Deduped MCP, get the setup steps for huggingface/peft and "Fine-tune a small LLM with LoRA on one GPU"

Add Deduped to Claude Code once

claude mcp add --transport http deduped https://dedupedai.com/mcp

Cursor, ChatGPT, others

Share: X LinkedIn Hacker News Email Copy link

Third-party repos we don't own. AI-written use cases can be wrong. Disclaimer