How to use TRL: what you can do with it

3 use cases

• Fine-tune a small model from the command line

  Run supervised fine-tuning on Qwen2.5-0.5B with a public dataset using only the TRL CLI.

  • REPO huggingface/trl
  • Claude Code
  • Codex
  • Cursor
  • Gemini CLI
  • Software and DevOps
  • Intermediate
  View prompt

  Using huggingface/trl, install it with `pip install trl`, run `trl sft --model_name_or_path Qwen/Qwen2.5-0.5B` with `--dataset_name trl-lib/Capybara` and `--output_dir Qwen2.5-0.5B-SFT`, and tell me how to load the result for a quick chat test.

  Share Email me this

• Align a model to preferences with DPO

  Train a model on chosen versus rejected answers with DPOTrainer or the TRL dpo command.

  • REPO huggingface/trl
  • Claude Code
  • Codex
  • Cursor
  • Gemini CLI
  • Software and DevOps
  • Advanced
  View prompt

  With huggingface/trl, train a DPO model using `trl dpo --model_name_or_path Qwen/Qwen2.5-0.5B-Instruct` and `--dataset_name argilla/Capybara-Preferences`, then write a small script that compares base and trained answers on 10 prompts from <my list>.

  Share Email me this

• Train math reasoning with GRPO

  Use GRPOTrainer with the built-in accuracy_reward on a math dataset, the method behind DeepSeek R1.

  • REPO huggingface/trl
  • Claude Code
  • Codex
  • Cursor
  • Gemini CLI
  • Software and DevOps
  • Advanced
  View prompt

  Using huggingface/trl, install `pip install trl[math_verify]`, write a GRPOTrainer script on trl-lib/DeepMath-103K with accuracy_reward and Qwen/Qwen2.5-0.5B-Instruct, and add logging so I can watch the reward climb.

  Share Email me this

Quick start with TRL

1. Check you have what it needs: Python with pip; A GPU is strongly recommended; the README covers NVIDIA, AMD and AWS Trainium setups; Access to Hugging Face Hub models and datasets.
2. Open huggingface/trl on GitHub and read the README and license.
3. Install TRL with pip install trl
4. Run trl sft --model_name_or_path Qwen/Qwen2.5-0.5B with --dataset_name trl-lib/Capybara and --output_dir Qwen2.5-0.5B-SFT
5. Wait for training to finish and check the output folder
6. Run TRL sft with --help to see more options

These steps come from our top use case, Fine-tune a small model from the command line, which also has a copy-paste agent prompt.

Using the Deduped MCP, get the setup steps for huggingface/trl

claude mcp add --transport http deduped https://dedupedai.com/mcp

Best AI repos for Software and DevOps

About this repo

• Who it is forML engineers and tinkerers who want to fine-tune or align open models with standard recipes.
• Why it is hot20k stars, talked about in 2 AI sources we track
• LanguagePython
• Sourcegithub.com/huggingface/trl · Back to Hot repos

Requirements

• Python with pip
• A GPU is strongly recommended; the README covers NVIDIA, AMD and AWS Trainium setups
• Access to Hugging Face Hub models and datasets

Related repos

• Trended earlier REPO huggingface/peft Peft PEFT is Hugging Face's library of parameter-efficient fine-tuning methods such as LoRA. Instead of training all of a model's weights, it trains a small adapter, often well under 1% of parameters, which cuts GPU memory and storage while keeping performance close to full fine-tuning. It integrates with Transformers, Diffusers, Accelerate and TRL, and combines with quantization for training on consumer GPUs.
• Trended earlier REPO whitecircle/halo Halo Halo is an open-source framework from White Circle for training large language and multimodal models with pre-training, SFT, preference optimization, and multi-turn RL on Hugging Face models.
• Trended earlier REPO aibuildai-inc/aibuildai-llm-posttrain-agent Aibuildai LLM Posttrain Agent A source-code release of a recursive self-improving agent that designs, runs, and iterates on post-training experiments for LLMs.