Agent skill

Grpo Rl Training

Expert guidance for GRPO/RL fine-tuning with TRL for reasoning and task-specific model training.

MITAuto-check passedAI & LLM Engineering

Install Grpo Rl Training

skills CLI

$ npx skills add AlexAI-MCP/hermes-CCC --skill grpo-rl-training -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI

$ gh skill install AlexAI-MCP/hermes-CCC grpo-rl-training --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy

$ git clone --depth 1 https://github.com/AlexAI-MCP/hermes-CCC.git skills-src && mkdir -p .claude/skills && cp -r skills-src/skills/grpo-rl-training .claude/skills/grpo-rl-training && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Paste into Claude Code

Install the "grpo-rl-training" agent skill from https://github.com/AlexAI-MCP/hermes-CCC/tree/master/skills/grpo-rl-training into .claude/skills/grpo-rl-training/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "grpo-rl-training", then confirm the skill loads.

Claude Code copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

$skill-installer

$skill-installer install https://github.com/AlexAI-MCP/hermes-CCC/tree/master/skills/grpo-rl-training

Type this inside Codex. $skill-installer <name> installs a curated skill from openai/skills. The installer writes to $CODEX_HOME/skills (default ~/.codex/skills). Restart Codex if the skill does not show up.

skills CLI

$ npx skills add AlexAI-MCP/hermes-CCC --skill grpo-rl-training -a codex

Project install goes to .agents/skills/; add -g for ~/.codex/skills/.

GitHub CLI

$ gh skill install AlexAI-MCP/hermes-CCC grpo-rl-training --agent codex

Project scope by default (.agents/skills/); add --scope user for a personal install.

Manual copy

$ git clone --depth 1 https://github.com/AlexAI-MCP/hermes-CCC.git skills-src && mkdir -p .agents/skills && cp -r skills-src/skills/grpo-rl-training .agents/skills/grpo-rl-training && rm -rf skills-src

Use ~/.agents/skills/ instead of .agents/skills for a personal install.

Codex skills documentation · loads skills from .agents/skills/

Paste into Codex

Install the "grpo-rl-training" agent skill from https://github.com/AlexAI-MCP/hermes-CCC/tree/master/skills/grpo-rl-training into .agents/skills/grpo-rl-training/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "grpo-rl-training", then confirm the skill loads.

Codex copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

skills CLI

$ npx skills add AlexAI-MCP/hermes-CCC --skill grpo-rl-training -a cursor

Project install goes to .agents/skills/; add -g for ~/.cursor/skills/.

GitHub CLI

$ gh skill install AlexAI-MCP/hermes-CCC grpo-rl-training --agent cursor

Project scope by default (.agents/skills/); add --scope user for a personal install.

Manual copy

$ git clone --depth 1 https://github.com/AlexAI-MCP/hermes-CCC.git skills-src && mkdir -p .cursor/skills && cp -r skills-src/skills/grpo-rl-training .cursor/skills/grpo-rl-training && rm -rf skills-src

Use ~/.cursor/skills/ instead of .cursor/skills for a personal install.

Cursor skills documentation · loads skills from .cursor/skills/, .agents/skills/, .claude/skills/, .codex/skills/

Paste into Cursor

Install the "grpo-rl-training" agent skill from https://github.com/AlexAI-MCP/hermes-CCC/tree/master/skills/grpo-rl-training into .cursor/skills/grpo-rl-training/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "grpo-rl-training", then confirm the skill loads.

Cursor copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

gemini skills install

$ gemini skills install https://github.com/AlexAI-MCP/hermes-CCC.git --path skills/grpo-rl-training

--scope user (default) or --scope workspace; --path is the subfolder of the repo that holds the skill; --consent skips the security confirmation prompt.

skills CLI

$ npx skills add AlexAI-MCP/hermes-CCC --skill grpo-rl-training -a gemini-cli

Project install goes to .agents/skills/; add -g for ~/.gemini/skills/.

GitHub CLI

$ gh skill install AlexAI-MCP/hermes-CCC grpo-rl-training --agent gemini-cli

Project scope by default (.agents/skills/); add --scope user for a personal install.

Manual copy

$ git clone --depth 1 https://github.com/AlexAI-MCP/hermes-CCC.git skills-src && mkdir -p .gemini/skills && cp -r skills-src/skills/grpo-rl-training .gemini/skills/grpo-rl-training && rm -rf skills-src

Use ~/.gemini/skills/ instead of .gemini/skills for a personal install, then run /skills reload.

Gemini CLI skills documentation · loads skills from .gemini/skills/, .agents/skills/

Paste into Gemini CLI

Install the "grpo-rl-training" agent skill from https://github.com/AlexAI-MCP/hermes-CCC/tree/master/skills/grpo-rl-training into .gemini/skills/grpo-rl-training/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "grpo-rl-training", then confirm the skill loads.

Gemini CLI copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

GitHub CLI

$ gh skill install AlexAI-MCP/hermes-CCC grpo-rl-training

Installs for Copilot at project scope by default; add --scope user for a personal install. Preview a skill first with gh skill preview. Needs GitHub CLI 2.90.0 or later (public preview).

skills CLI

$ npx skills add AlexAI-MCP/hermes-CCC --skill grpo-rl-training -a github-copilot

Project install goes to .agents/skills/; add -g for ~/.copilot/skills/.

Manual copy

$ git clone --depth 1 https://github.com/AlexAI-MCP/hermes-CCC.git skills-src && mkdir -p .github/skills && cp -r skills-src/skills/grpo-rl-training .github/skills/grpo-rl-training && rm -rf skills-src

Use ~/.copilot/skills/ instead of .github/skills for a personal install. Commit .github/skills so cloud agent and code review can use it.

GitHub Copilot skills documentation · loads skills from .github/skills/, .claude/skills/, .agents/skills/

Paste into GitHub Copilot

Install the "grpo-rl-training" agent skill from https://github.com/AlexAI-MCP/hermes-CCC/tree/master/skills/grpo-rl-training into .github/skills/grpo-rl-training/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "grpo-rl-training", then confirm the skill loads.

GitHub Copilot copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

skills CLI

$ npx skills add AlexAI-MCP/hermes-CCC --skill grpo-rl-training -a opencode

OpenCode documents no install command of its own. Project install goes to .agents/skills/; add -g for ~/.config/opencode/skills/.

GitHub CLI

$ gh skill install AlexAI-MCP/hermes-CCC grpo-rl-training --agent opencode

Project scope by default (.agents/skills/); add --scope user for a personal install.

Manual copy

$ git clone --depth 1 https://github.com/AlexAI-MCP/hermes-CCC.git skills-src && mkdir -p .opencode/skills && cp -r skills-src/skills/grpo-rl-training .opencode/skills/grpo-rl-training && rm -rf skills-src

Use ~/.config/opencode/skills/ instead of .opencode/skills for a personal install.

OpenCode skills documentation · loads skills from .opencode/skills/, .claude/skills/, .agents/skills/

Paste into OpenCode

Install the "grpo-rl-training" agent skill from https://github.com/AlexAI-MCP/hermes-CCC/tree/master/skills/grpo-rl-training into .opencode/skills/grpo-rl-training/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "grpo-rl-training", then confirm the skill loads.

OpenCode copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

Facts

Skill name: grpo-rl-training
GitHub stars: 135
Token cost: ~1.3k tokens
SKILL.md length: 226 words
Files: 1
Skills in repo: 44
Repo updated: Apr 8, 2026
First seen: Oct 7, 2026
Licence: MIT

At a glance

Expert guidance for GRPO/RL fine-tuning with TRL for reasoning and task-specific model training.

Tasks that involve Fine-tuning
SKILL.md covers When to Use, Setup, Minimal GRPO Example and Reward Function Patterns, plus 5 more sections
Calls pip

What it does

Grpo Rl Training is an agent skill from AlexAI-MCP/hermes-CCC. Expert guidance for GRPO/RL fine-tuning with TRL for reasoning and task-specific model training.

Its SKILL.md is about 1.3k tokens, which your agent loads only when the skill is triggered. It is a single SKILL.md file with no bundled scripts.

It sits in AI & LLM Engineering, covering Fine-tuning. The repository describes itself as: Hermes Agent ported to Claude Code Channel — 46 native skills, no OAuth, no external process. The licence is MIT.

When your agent uses it

Tasks that involve Fine-tuning

Example prompts

“/grpo-rl-training”

Requirements

Python 3

What it can do on your machine

Read from SKILL.md and the folder at commit 8107e89. It shows what the files ask for, not the result of running them.

Tool permissions
Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.
From allowed-tools in the SKILL.md frontmatter.
Runs code
Shell commands in SKILL.md call:
- pip
From the folder's file list and the shell code blocks in SKILL.md.
Network
No URLs in SKILL.md. Its commands use pip, which can reach the network depending on how they are called.
From URLs in SKILL.md, links to its own repository left out.
Credentials
Names no API keys, tokens, secrets or passwords.
From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Grpo Rl Training loads about 1.3k tokens when it runs. Until then it costs about 28 tokens; SKILL.md has 226 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it: ~28
When it runs · the whole SKILL.md, loaded when a task matches: ~1.3k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from AlexAI-MCP/hermes-CCC at commit 8107e89, republished under its MIT licence (© AlexAI-MCP). 226 words, ~1,286 tokens.

Download SKILL.mdSave it as .claude/skills/grpo-rl-training/SKILL.md (or your agent's skills folder).

name: grpo-rl-training
description: Expert guidance for GRPO/RL fine-tuning with TRL for reasoning and task-specific model training.
version: 1.0.0
author: hermes-CCC (ported from Hermes Agent by NousResearch)
license: MIT

GRPO/RL Training with TRL

Expert guidance for Group Relative Policy Optimization (GRPO) using the TRL library. Battle-tested patterns for fine-tuning language models with custom reward functions.

When to Use

Teaching a model to follow specific output formats (JSON, structured reasoning)
Improving accuracy on math, coding, or reasoning tasks
Custom task-specific behavior without labeled datasets
Distilling reasoning capabilities from larger models

Setup

bash

pip install transformers>=4.47.0 trl>=0.14.0 datasets>=3.2.0 peft>=0.14.0 torch accelerate

Optional for logging:

bash

pip install wandb
wandb login

Minimal GRPO Example

python

from trl import GRPOConfig, GRPOTrainer
from transformers import AutoModelForCausalLM, AutoTokenizer
from datasets import load_dataset

model_name = "Qwen/Qwen2.5-1.5B-Instruct"
model = AutoModelForCausalLM.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

dataset = load_dataset("your/dataset")

# Define reward function
def reward_fn(prompts, completions, **kwargs):
    """
    Returns list of floats — one reward per completion.
    Higher = better. Typically in range [-1, 1] or [0, 1].
    """
    rewards = []
    for completion in completions:
        # Example: reward for correct format
        if completion.strip().startswith("<answer>"):
            rewards.append(1.0)
        else:
            rewards.append(-0.5)
    return rewards

config = GRPOConfig(
    output_dir="./grpo-output",
    learning_rate=5e-6,
    num_train_epochs=3,
    per_device_train_batch_size=4,
    gradient_accumulation_steps=4,
    num_generations=8,        # completions per prompt (G in GRPO)
    max_prompt_length=512,
    max_completion_length=256,
    kl_coef=0.1,              # KL penalty — start low, tune up if reward hacking
    logging_steps=10,
    save_steps=100,
    report_to="wandb",        # or "none"
)

trainer = GRPOTrainer(
    model=model,
    args=config,
    reward_funcs=reward_fn,
    train_dataset=dataset["train"],
    tokenizer=tokenizer,
)

trainer.train()
trainer.save_model("./grpo-final")

Reward Function Patterns

Format Reward

python

import re

def format_reward(prompts, completions, **kwargs):
    pattern = r"<think>.*?</think>\s*<answer>.*?</answer>"
    return [1.0 if re.fullmatch(pattern, c, re.DOTALL) else -1.0 for c in completions]

Accuracy Reward

python

def accuracy_reward(prompts, completions, ground_truth, **kwargs):
    rewards = []
    for completion, gt in zip(completions, ground_truth):
        extracted = extract_answer(completion)
        rewards.append(1.0 if extracted == gt else 0.0)
    return rewards

Length Penalty

python

def length_penalty_reward(prompts, completions, **kwargs):
    return [max(0, 1.0 - len(c) / 2000) for c in completions]

Combined Rewards

python

reward_funcs = [format_reward, accuracy_reward]  # TRL averages them

PEFT/LoRA Integration (Memory Efficient)

python

from peft import LoraConfig, get_peft_model

lora_config = LoraConfig(
    r=16,
    lora_alpha=32,
    target_modules=["q_proj", "v_proj"],
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM",
)

model = get_peft_model(model, lora_config)
model.print_trainable_parameters()

Key Hyperparameters

Param	Default	Effect
`num_generations`	8	More = better gradient estimate, more memory
`kl_coef`	0.1	Higher = stays closer to reference model
`learning_rate`	5e-6	Lower than SFT — RL is unstable with high LR
`max_completion_length`	256	Controls max tokens generated

Common Pitfalls

Reward Hacking: Model finds shortcuts to maximize reward without actually improving. → Fix: Add KL penalty (kl_coef), diverse reward signals, human evaluation

Mode Collapse: All completions become similar. → Fix: Increase num_generations, add temperature, check reward diversity

OOM: Large models with many generations blow up memory. → Fix: Use LoRA, reduce num_generations, use gradient_checkpointing=True

Reward too sparse: Model rarely gets positive reward → no learning signal. → Fix: Start with easier examples, use shaped reward (partial credit)

Monitoring with W&B

Key metrics to watch:

train/reward: should increase over time
train/kl: should stay bounded (spike = reward hacking)
train/policy_loss: learning signal
train/entropy: diversity of completions (drop = mode collapse)

Checkpoint and Resume

bash

# Resume from checkpoint
trainer = GRPOTrainer(..., resume_from_checkpoint="./grpo-output/checkpoint-500")
trainer.train(resume_from_checkpoint=True)

Files

Just SKILL.md in skills/grpo-rl-training of AlexAI-MCP/hermes-CCC.

Open the folder on GitHubat commit 8107e89

Compare with similar skills

Grpo Rl Training next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Grpo Rl Training compared with similar skills
Skill	Stars	Used in	Tokens	Auto-check	Licence	Repo updated
Grpo Rl Training this skillAlexAI-MCP/hermes-CCC	135	—	~1.3k	Automated safety check: Pass	MIT	6 mo ago
Sentence-Transformers Training Routerhuggingface/skills	11k	1 repos	~2.6k	Automated safety check: Pass	Apache-2.0	2 days ago
Train RlOpenPipe/ART	11k	—	~2.4k	Automated safety check: Pass	Apache-2.0	today
Qwopus27b Rl TrainingR6410418/Jackrong-llm-finetuning-guide	1.7k	—	~830	Automated safety check: Pass	Apache-2.0	3 mo ago
Dataset Evaluationawslabs/agent-plugins	916	1 repos	~1.3k	Automated safety check: Pass	Apache-2.0	yesterday
Train SftOpenPipe/ART	11k	—	~2.9k	Automated safety check: Pass	Apache-2.0	today

Similar skills

Sentence-Transformers Training Router
huggingface/skills
Official
Routes a sentence-transformers training task to the right model type and required reference docs and example scripts, covering bi-encoders, rerankers, sparse and multi-vector models.
11k GitHub starsUsed in 1 repo~2.6k tokens
AI & LLM EngineeringAuto-check passed
Train Rl
OpenPipe/ART
RL training reference for the ART framework. An agent skill from OpenPipe/ART.
11k GitHub stars~2.4k tokensUpdated today
AI & LLM EngineeringAuto-check passed
Qwopus27b Rl Training
R6410418/Jackrong-llm-finetuning-guide
Prepare, validate, launch-plan, monitor, resume, and stop configurable Qwopus 27B reinforcement-learning workflows for GRPO or GSPO.
1.7k GitHub stars~830 tokensUpdated 3 mo ago
AI & LLM EngineeringAuto-check passed
Dataset Evaluation
awslabs/agent-plugins
Official
Validates dataset formatting and quality for SageMaker model fine-tuning (SFT, DPO, or RLVR).
916 GitHub starsUsed in 1 repo~1.3k tokens
AI & LLM EngineeringAuto-check passed
Train Sft
OpenPipe/ART
SFT training reference for the ART framework. An agent skill from OpenPipe/ART.
11k GitHub stars~2.9k tokensUpdated today
AI & LLM EngineeringAuto-check passed
Fine Tuning With Trl
Orchestra-Research/AI-Research-SKILLs
Fine-tune LLMs using reinforcement learning with TRL - SFT for instruction tuning, DPO for preference alignment, PPO/GRPO for reward optimization, and reward model training.
13k GitHub starsUsed in 6 repos~2.9k tokens
AI & LLM EngineeringAuto-check passed

Questions about Grpo Rl Training

What does Grpo Rl Training do?

Expert guidance for GRPO/RL fine-tuning with TRL for reasoning and task-specific model training. Grpo Rl Training is an agent skill from AlexAI-MCP/hermes-CCC. Expert guidance for GRPO/RL fine-tuning with TRL for reasoning and task-specific model training.

When should I use Grpo Rl Training?

Grpo Rl Training fits situations like: tasks that involve Fine-tuning.

How do I install Grpo Rl Training in Claude Code?

Run `npx skills add AlexAI-MCP/hermes-CCC --skill grpo-rl-training -a claude-code`. Or copy the skill folder (skills/grpo-rl-training in AlexAI-MCP/hermes-CCC) into .claude/skills/grpo-rl-training in your project. Claude Code loads it when a task matches its description.

How do I install Grpo Rl Training in Codex?

Run `npx skills add AlexAI-MCP/hermes-CCC --skill grpo-rl-training -a codex`. Or copy the skill folder (skills/grpo-rl-training in AlexAI-MCP/hermes-CCC) into .agents/skills/grpo-rl-training in your project. Codex loads it when a task matches its description.

Can I use Grpo Rl Training in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add AlexAI-MCP/hermes-CCC --skill grpo-rl-training -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/grpo-rl-training, .gemini/skills/grpo-rl-training, .github/skills/grpo-rl-training and .opencode/skills/grpo-rl-training in your project.

What does Grpo Rl Training need to run?

Going by SKILL.md and its folder, Grpo Rl Training needs the command-line tools its instructions call (pip). Our summary lists: Python 3.

Does Grpo Rl Training access the network?

SKILL.md contains no URLs. Its commands use pip, which can reach the network depending on how they are called. This is read from the text; nothing was executed.

Is Grpo Rl Training safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.

What licence does Grpo Rl Training use?

Grpo Rl Training is published under the MIT licence (declared in SKILL.md). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Grpo Rl Training use?

About 1.3k tokens (SKILL.md is roughly 5.1k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.

What are the alternatives to Grpo Rl Training?

Skills that share tags, products or a category with Grpo Rl Training: Sentence-Transformers Training Router (huggingface/skills, 11k stars), Train Rl (OpenPipe/ART, 11k stars), Qwopus27b Rl Training (R6410418/Jackrong-llm-finetuning-guide, 1.7k stars) and Dataset Evaluation (awslabs/agent-plugins, 916 stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Grpo Rl Training?

AlexAI-MCP (a GitHub user) maintains it in AlexAI-MCP/hermes-CCC, which has 135 GitHub stars. The repository holds 44 skills in this directory. The repository was last updated on April 8, 2026.

Source: AlexAI-MCP/hermes-CCC on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.

Install Grpo Rl Training

Facts

At a glance

What it does

When your agent uses it

Example prompts

Requirements

What it can do on your machine

Tool permissions

Runs code

Network

Credentials

Context cost

Safety

SKILL.md

GRPO/RL Training with TRL

When to Use

Setup

Minimal GRPO Example

Reward Function Patterns

Format Reward

Accuracy Reward

Length Penalty

Combined Rewards

PEFT/LoRA Integration (Memory Efficient)

Key Hyperparameters

Common Pitfalls

Monitoring with W&B

Checkpoint and Resume

Files

Compare with similar skills

Similar skills

More from AlexAI-MCP/hermes-CCC

Related

Questions about Grpo Rl Training