New: 98,000+ skills indexed from GitHub repos with 100+ stars · every skill safety-checkedBrowse top skillsTop skills
Esc to closeBrowse all skills
Submit a skillBrowse skills
Submit a skillBrowse skills

Agent skill

Trl Training

by waybarrios in waybarrios/opencode-power-pack

Train and fine-tune transformer language models using TRL (Transformers Reinforcement Learning).

Apache-2.0Auto-check passedAI & LLM Engineering

Install Trl Training

skills CLI
$ npx skills add waybarrios/opencode-power-pack --skill trl-training -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install waybarrios/opencode-power-pack trl-training --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/waybarrios/opencode-power-pack.git skills-src && mkdir -p .claude/skills && cp -r skills-src/skills/trl-training .claude/skills/trl-training && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
trl-training
GitHub stars
533
Used in
3 other repos
Token cost
~2.1k tokens
SKILL.md length
558 words
Files
1
Skills in repo
32
Repo updated
First seen
Licence
Apache-2.0

At a glance

Train and fine-tune transformer language models using TRL (Transformers Reinforcement Learning).

  • Works in 7 steps: Start with SFT: Always fine-tune base… → Use LoRA for efficiency: Enable… → Monitor training: Use --report_to… → …
  • Tasks that involve Fine-tuning
  • SKILL.md covers Overview, Core Commands, Configuration Files and Distributed Training, plus 3 more sections
  • Calls hf

What it does

Trl Training is an agent skill from waybarrios/opencode-power-pack. Train and fine-tune transformer language models using TRL (Transformers Reinforcement Learning). Supports SFT, DPO, GRPO, KTO, RLOO and Reward Model training via CLI commands.

Its SKILL.md is about 2.1k tokens, which your agent loads only when the skill is triggered. It is a single SKILL.md file with no bundled scripts.

It sits in AI & LLM Engineering, covering Fine-tuning and Reinforcement learning. The repository describes itself as: 54 rigorous skills for Codex, OpenCode, and Pi: code review, security audit, feature development, frontend design, MCP tools, Hugging Face ML/training, and more. The licence is Apache-2.0.

When your agent uses it

  • Tasks that involve Fine-tuning
  • Tasks that involve Reinforcement learning

Example prompts

  • “/trl-training”

Workflow steps

7 steps, taken from the first numbered list in SKILL.md.

  1. Start with SFT: Always fine-tune base models with SFT before preference alignment
  2. Use LoRA for efficiency: Enable --use_peft for faster training and lower memory
  3. Monitor training: Use --report_to trackio (or --report_to wandb or --report_to tensorboard) for tracking
  4. Save checkpoints: TRL automatically saves checkpoints in --output_dir
  5. Test on small datasets first: Verify pipeline works before full training
  6. Use configuration files: Create YAML configs for reproducibility
  7. Leverage Accelerate: Use multi-GPU training for faster iteration

What it can do on your machine

Read from SKILL.md and the folder at commit 9dccb6d. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    Shell commands in SKILL.md call:

    • hf

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    Links to these hosts (documentation or services it may open):

    • github.com
    • huggingface.co

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Trl Training loads about 2.1k tokens when it runs. Until then it costs about 47 tokens; SKILL.md has 558 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~47
When it runs · the whole SKILL.md, loaded when a task matches
~2.1k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from waybarrios/opencode-power-pack at commit 9dccb6d, republished under its Apache-2.0 licence (© waybarrios). 558 words, ~2,126 tokens.

Download SKILL.mdSave it as .claude/skills/trl-training/SKILL.md (or your agent's skills folder).
name
trl-training
description
Train and fine-tune transformer language models using TRL (Transformers Reinforcement Learning). Supports SFT, DPO, GRPO, KTO, RLOO and Reward Model training via CLI commands.
license
Apache-2.0 (modified; see UPSTREAMS.json)

TRL Training Skill

You are an expert at using the TRL (Transformers Reinforcement Learning) library to train and fine-tune large language models.

Overview

TRL provides CLI commands for post-training foundation models using state-of-the-art techniques:

  • SFT (Supervised Fine-Tuning): Fine-tune models on instruction-following or conversational datasets
  • DPO (Direct Preference Optimization): Align models using preference data
  • GRPO (Group Relative Policy Optimization): Train models by ranking multiple sampled outputs relative to each other and optimizing based on their comparative rewards.
  • RLOO (Reinforce Leave One Out): Online RL training with generation-based rewards
  • Reward Model Training: Train reward models for RLHF

TRL is built on top of Hugging Face Transformers and Accelerate, providing seamless integration with the Hugging Face ecosystem.

Core Commands

trl sft - Supervised Fine-Tuning

Fine-tune language models on instruction-following or conversational datasets.

Full training:

bash
trl sft \
  --model_name_or_path Qwen/Qwen2-0.5B \
  --dataset_name trl-lib/Capybara \
  --learning_rate 2.0e-5 \
  --num_train_epochs 1 \
  --packing \
  --per_device_train_batch_size 2 \
  --gradient_accumulation_steps 8 \
  --eos_token '<|im_end|>' \
  --eval_strategy steps \
  --eval_steps 100 \
  --output_dir Qwen2-0.5B-SFT \
  --push_to_hub

Train with LoRA adapters:

bash
trl sft \
  --model_name_or_path Qwen/Qwen2-0.5B \
  --dataset_name trl-lib/Capybara \
  --learning_rate 2.0e-4 \
  --num_train_epochs 1 \
  --packing \
  --per_device_train_batch_size 2 \
  --gradient_accumulation_steps 8 \
  --eos_token '<|im_end|>' \
  --eval_strategy steps \
  --eval_steps 100 \
  --use_peft \
  --lora_r 32 \
  --lora_alpha 16 \
  --output_dir Qwen2-0.5B-SFT \
  --push_to_hub
trl dpo - Direct Preference Optimization

Align models using preference data (chosen/rejected pairs).

Full training:

bash
trl dpo \
  --dataset_name trl-lib/ultrafeedback_binarized \
  --model_name_or_path Qwen/Qwen2-0.5B-Instruct \
  --learning_rate 5.0e-7 \
  --num_train_epochs 1 \
  --per_device_train_batch_size 2 \
  --max_steps 1000 \
  --gradient_accumulation_steps 8 \
  --eval_strategy steps \
  --eval_steps 50 \
  --output_dir Qwen2-0.5B-DPO \
  --no_remove_unused_columns

Train with LoRA adapters:

bash
trl dpo \
  --dataset_name trl-lib/ultrafeedback_binarized \
  --model_name_or_path Qwen/Qwen2-0.5B-Instruct \
  --learning_rate 5.0e-6 \
  --num_train_epochs 1 \
  --per_device_train_batch_size 2 \
  --max_steps 1000 \
  --gradient_accumulation_steps 8 \
  --eval_strategy steps \
  --eval_steps 50 \
  --output_dir Qwen2-0.5B-DPO \
  --no_remove_unused_columns \
  --use_peft \
  --lora_r 32 \
  --lora_alpha 16
trl grpo - Group Relative Policy Optimization

Train models using reward functions or LLM-as-a-judge for evaluating generations and providing rewards.

Basic usage:

bash
trl grpo \
  --model_name_or_path Qwen/Qwen2.5-0.5B \
  --dataset_name trl-lib/gsm8k \
  --reward_funcs accuracy_reward \
  --output_dir Qwen2-0.5B-GRPO \
  --push_to_hub
trl rloo - Reinforce Leave One Out

Online RL training where the model generates text and receives rewards based on custom criteria.

Basic usage:

bash
trl rloo \
  --model_name_or_path Qwen/Qwen2.5-0.5B \
  --dataset_name trl-lib/tldr \
  --reward_model_name_or_path sentiment-analysis:nlptown/bert-base-multilingual-uncased-sentiment \
  --output_dir Qwen2-0.5B-RLOO \
  --push_to_hub
trl reward - Reward Model Training

Train a reward model to score text quality for RLHF.

Full training:

bash
trl reward \
  --model_name_or_path Qwen/Qwen2-0.5B-Instruct \
  --dataset_name trl-lib/ultrafeedback_binarized \
  --output_dir Qwen2-0.5B-Reward \
  --per_device_train_batch_size 8 \
  --num_train_epochs 1 \
  --learning_rate 1.0e-5 \
  --eval_strategy steps \
  --eval_steps 50 \
  --max_length 2048

Train with LoRA adapters:

bash
trl reward \
  --model_name_or_path Qwen/Qwen2-0.5B-Instruct \
  --dataset_name trl-lib/ultrafeedback_binarized \
  --output_dir Qwen2-0.5B-Reward-LoRA \
  --per_device_train_batch_size 8 \
  --num_train_epochs 1 \
  --learning_rate 1.0e-4 \
  --eval_strategy steps \
  --eval_steps 50 \
  --max_length 2048 \
  --use_peft \
  --lora_task_type SEQ_CLS \
  --lora_r 32 \
  --lora_alpha 16

Configuration Files

TRL supports YAML configuration files for reproducible training. All CLI arguments can be specified in a config file.

Example config (sft_config.yaml):

yaml
model_name_or_path: Qwen/Qwen2.5-0.5B
dataset_name: trl-lib/Capybara
learning_rate: 2.0e-5
num_train_epochs: 1
per_device_train_batch_size: 8
gradient_accumulation_steps: 2
output_dir: ./sft_output
use_peft: true
lora_r: 16
lora_alpha: 16
report_to: trackio

Launch with config:

bash
trl sft --config sft_config.yaml

Override config values:

bash
trl sft --config sft_config.yaml --learning_rate 1.0e-5

Distributed Training

TRL integrates with Accelerate for multi-GPU and multi-node training.

Multi-GPU training:

bash
trl sft \
  --config sft_config.yaml \
  --num_processes 4

Use predefined Accelerate configs:

TRL provides predefined configs: single_gpu, multi_gpu, fsdp1, fsdp2, zero1, zero2, zero3

bash
trl sft \
  --config sft_config.yaml \
  --accelerate_config zero2

Custom Accelerate config:

bash
# Generate custom config
accelerate config

# Use custom config
trl sft --config sft_config.yaml --config_file ~/.cache/huggingface/accelerate/default_config.yaml

Fully Sharded Data Parallel (FSDP):

bash
trl sft --config sft_config.yaml --accelerate_config fsdp2

DeepSpeed ZeRO:

bash
trl sft --config sft_config.yaml --accelerate_config zero3

Troubleshooting

CUDA Out of Memory
  • Reduce --per_device_train_batch_size and increase --gradient_accumulation_steps
  • Enable --use_peft for LoRA training
  • Use --gradient_checkpointing to save memory
  • Try smaller model or longer sequence truncation
Dataset Loading Issues
  • Verify dataset exists: check Hugging Face Hub or local path
  • Check dataset format matches expected columns
  • Use --dataset_config for multi-config datasets
  • Inspect dataset: from datasets import load_dataset; ds = load_dataset(name)
Show full SKILL.md (215 more words)Show less
Model Loading Issues
  • Verify model exists on Hugging Face Hub
  • Check if gated model requires authentication: hf auth login
  • For local models, provide absolute path
  • Ensure sufficient disk space and memory
Slow Training
  • Enable dataset --packing for short sequences
  • Use larger --per_device_train_batch_size if memory allows
  • Enable --tf32 for faster computation on Ampere GPUs
  • Use --bf16 on supported hardware
  • Consider multi-GPU training with --num_processes
Generation Issues (GRPO/RLOO)
  • Check prompt format in dataset
  • Adjust --temperature and --top_p for generation
  • Verify the reward function (for GRPO/RLOO)

Additional Resources

Best Practices

  1. Start with SFT: Always fine-tune base models with SFT before preference alignment
  2. Use LoRA for efficiency: Enable --use_peft for faster training and lower memory
  3. Monitor training: Use --report_to trackio (or --report_to wandb or --report_to tensorboard) for tracking
  4. Save checkpoints: TRL automatically saves checkpoints in --output_dir
  5. Test on small datasets first: Verify pipeline works before full training
  6. Use configuration files: Create YAML configs for reproducibility
  7. Leverage Accelerate: Use multi-GPU training for faster iteration

When helping users with TRL:

  • Always check which training method is appropriate for their use case
  • Verify dataset format matches the expected schema
  • Recommend starting with smaller models for testing
  • Suggest LoRA for resource-constrained environments
  • Point to specific documentation sections for advanced features

© waybarrios, Apache-2.0. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

Just SKILL.md in skills/trl-training of waybarrios/opencode-power-pack.

Open the folder on GitHubat commit 9dccb6d

Used in 3 other repositories

We found 8 copies of this SKILL.md (exact, near-identical or edited) in other folders, from 3 other GitHub owners. This page covers the copy in waybarrios/opencode-power-pack, which our catalogue first saw on October 7, 2026.

Compare with similar skills

Trl Training next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Trl Training compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Trl Training this skillwaybarrios/opencode-power-pack5333 repos~2.1kAutomated safety check: PassApache-2.0
Hugging Face LLM Trainerhuggingface/skills11k3 repos~7.2kAutomated safety check: PassApache-2.0
Train RlOpenPipe/ART11k—~2.4kAutomated safety check: PassApache-2.0
Qwopus27b Rl TrainingR6410418/Jackrong-llm-finetuning-guide1.7k—~830Automated safety check: PassApache-2.0
Fine Tuning With TrlOrchestra-Research/AI-Research-SKILLs13k7 repos~2.9kAutomated safety check: PassMIT
Optim AgentOptim-Agent/optim-agent801—~1.3kAutomated safety check: PassMIT

Similar skills

  • Hugging Face LLM Trainer

    huggingface/skills

    Official

    Trains or fine-tunes language and vision models with TRL or Unsloth on Hugging Face Jobs cloud GPUs, then converts the results to GGUF.

    11k GitHub starsUsed in 3 repos~7.2k tokens
    AI & LLM EngineeringAuto-check passed

  • Train Rl

    OpenPipe/ART

    RL training reference for the ART framework. An agent skill from OpenPipe/ART.

    11k GitHub stars~2.4k tokensUpdated yesterday
    AI & LLM EngineeringAuto-check passed

  • Qwopus27b Rl Training

    R6410418/Jackrong-llm-finetuning-guide

    Prepare, validate, launch-plan, monitor, resume, and stop configurable Qwopus 27B reinforcement-learning workflows for GRPO or GSPO.

    1.7k GitHub stars~830 tokensUpdated 2 mo ago
    AI & LLM EngineeringAuto-check passed

  • Fine Tuning With Trl

    Orchestra-Research/AI-Research-SKILLs

    Fine-tune LLMs using reinforcement learning with TRL - SFT for instruction tuning, DPO for preference alignment, PPO/GRPO for reward optimization, and reward model training.

    13k GitHub starsUsed in 7 repos~2.9k tokens
    AI & LLM EngineeringAuto-check passed

  • Optim Agent

    Optim-Agent/optim-agent

    A skill your agent uses when the user wants to optimize configurable system parameters against a measurable scalar objective, especially for model training, inference, quantitative strategies…

    801 GitHub stars~1.3k tokensUpdated 1 mo ago
    AI & LLM EngineeringAuto-check passed

  • Safactory Workflows

    AI45Lab/SAfactory

    Integrate a benchmark or custom environment into SAfactory using fixed adapter templates and local contract tests, optionally run Docker/RJob evaluation, or prepare GRPO/RL training.

    236 GitHub stars~1.8k tokensUpdated 14 days ago
    AI & LLM EngineeringAuto-check passed

More from waybarrios/opencode-power-pack

All 32 skills in this repo

  • Hf Cloud Sagemaker Iam Preflight

    waybarrios/opencode-power-pack

    Verify or select a SageMaker execution role before creating models, endpoints, or training jobs.

    533 GitHub stars~1.6k tokensUpdated 2 days ago
    Auto-check passed

  • Huggingface LLM Trainer

    waybarrios/opencode-power-pack

    Train or fine-tune language models with TRL or Unsloth on Hugging Face Jobs, including SFT, DPO, GRPO, reward models, and GGUF conversion.

    533 GitHub stars~3k tokensUpdated 2 days ago
    Auto-check passed

  • Huggingface Vision Trainer

    waybarrios/opencode-power-pack

    Train object-detection, image-classification, or SAM segmentation models on Hugging Face Jobs.

    533 GitHub stars~2.7k tokensUpdated 2 days ago
    Auto-check passed

  • Codeql

    waybarrios/opencode-power-pack

    Run CodeQL database creation and security queries, add data-extension models, or process CodeQL SARIF.

    533 GitHub starsUsed in 2 repos~3.7k tokens
    Auto-check passed

  • Semgrep

    waybarrios/opencode-power-pack

    Run Semgrep static analysis across a codebase, optionally using Semgrep Pro for cross-file taint analysis.

    533 GitHub stars~2.4k tokensUpdated 2 days ago
    Auto-check passed

  • Insecure Defaults

    waybarrios/opencode-power-pack

    Detects fail-open insecure defaults (hardcoded secrets, weak auth, permissive security) that allow apps to run insecurely in production.

    533 GitHub starsUsed in 2 repos~1.3k tokens
    Auto-check passed

Categories

Questions about Trl Training

What does Trl Training do?

Train and fine-tune transformer language models using TRL (Transformers Reinforcement Learning). Trl Training is an agent skill from waybarrios/opencode-power-pack. Train and fine-tune transformer language models using TRL (Transformers Reinforcement Learning).

When should I use Trl Training?

Trl Training fits situations like: tasks that involve Fine-tuning; tasks that involve Reinforcement learning.

How do I install Trl Training in Claude Code?

Run `npx skills add waybarrios/opencode-power-pack --skill trl-training -a claude-code`. Or copy the skill folder (skills/trl-training in waybarrios/opencode-power-pack) into .claude/skills/trl-training in your project. Claude Code loads it when a task matches its description.

How do I install Trl Training in Codex?

Run `npx skills add waybarrios/opencode-power-pack --skill trl-training -a codex`. Or copy the skill folder (skills/trl-training in waybarrios/opencode-power-pack) into .agents/skills/trl-training in your project. Codex loads it when a task matches its description.

Can I use Trl Training in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add waybarrios/opencode-power-pack --skill trl-training -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/trl-training, .gemini/skills/trl-training, .github/skills/trl-training and .opencode/skills/trl-training in your project.

What does Trl Training need to run?

Going by SKILL.md and its folder, Trl Training needs the command-line tools its instructions call (hf).

Does Trl Training access the network?

SKILL.md names 2 domains. As links in the text: github.com and huggingface.co. This is read from the text; nothing was executed.

Is Trl Training safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.

What licence does Trl Training use?

Trl Training is published under the Apache-2.0 licence (declared in SKILL.md). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Trl Training use?

About 2.1k tokens (SKILL.md is roughly 8.5k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.

What are the alternatives to Trl Training?

Skills that share tags, products or a category with Trl Training: Hugging Face LLM Trainer (huggingface/skills, 11k stars), Train Rl (OpenPipe/ART, 11k stars), Qwopus27b Rl Training (R6410418/Jackrong-llm-finetuning-guide, 1.7k stars) and Fine Tuning With Trl (Orchestra-Research/AI-Research-SKILLs, 13k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Trl Training?

waybarrios (a GitHub user) maintains it in waybarrios/opencode-power-pack, which has 533 GitHub stars. The repository holds 32 skills in this directory. The repository was last updated on October 6, 2026.

Source: waybarrios/opencode-power-pack on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.