Magpie Kernel Evaluator
amd/skills
Benchmarks LLM inference and drives GPU kernel optimization with Magpie.
Guides reinforcement-learning research with torchforge, Meta's PyTorch-native library that keeps RL algorithms apart from infrastructure, including GRPO math-reasoning runs.
$ npx skills add Orchestra-Research/AI-Research-SKILLs --skill torchforge-rl-training -a claude-codeProject install by default; add -g for ~/.claude/skills/.
$ gh skill install Orchestra-Research/AI-Research-SKILLs torchforge-rl-training --agent claude-codeProject scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).
$ git clone --depth 1 https://github.com/Orchestra-Research/AI-Research-SKILLs.git skills-src && mkdir -p .claude/skills && cp -r skills-src/06-post-training/torchforge .claude/skills/torchforge-rl-training && rm -rf skills-srcUse ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.
Claude Code skills documentation · loads skills from .claude/skills/
Install the "torchforge-rl-training" agent skill from https://github.com/Orchestra-Research/AI-Research-SKILLs/tree/main/06-post-training/torchforge into .claude/skills/torchforge-rl-training/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "torchforge-rl-training", then confirm the skill loads.Claude Code copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$skill-installer install https://github.com/Orchestra-Research/AI-Research-SKILLs/tree/main/06-post-training/torchforgeType this inside Codex. $skill-installer <name> installs a curated skill from openai/skills. The installer writes to $CODEX_HOME/skills (default ~/.codex/skills). Restart Codex if the skill does not show up.
$ npx skills add Orchestra-Research/AI-Research-SKILLs --skill torchforge-rl-training -a codexProject install goes to .agents/skills/; add -g for ~/.codex/skills/.
$ gh skill install Orchestra-Research/AI-Research-SKILLs torchforge-rl-training --agent codexProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/Orchestra-Research/AI-Research-SKILLs.git skills-src && mkdir -p .agents/skills && cp -r skills-src/06-post-training/torchforge .agents/skills/torchforge-rl-training && rm -rf skills-srcUse ~/.agents/skills/ instead of .agents/skills for a personal install.
Codex skills documentation · loads skills from .agents/skills/
Install the "torchforge-rl-training" agent skill from https://github.com/Orchestra-Research/AI-Research-SKILLs/tree/main/06-post-training/torchforge into .agents/skills/torchforge-rl-training/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "torchforge-rl-training", then confirm the skill loads.Codex copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add Orchestra-Research/AI-Research-SKILLs --skill torchforge-rl-training -a cursorProject install goes to .agents/skills/; add -g for ~/.cursor/skills/.
$ gh skill install Orchestra-Research/AI-Research-SKILLs torchforge-rl-training --agent cursorProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/Orchestra-Research/AI-Research-SKILLs.git skills-src && mkdir -p .cursor/skills && cp -r skills-src/06-post-training/torchforge .cursor/skills/torchforge-rl-training && rm -rf skills-srcUse ~/.cursor/skills/ instead of .cursor/skills for a personal install.
Cursor skills documentation · loads skills from .cursor/skills/, .agents/skills/, .claude/skills/, .codex/skills/
Install the "torchforge-rl-training" agent skill from https://github.com/Orchestra-Research/AI-Research-SKILLs/tree/main/06-post-training/torchforge into .cursor/skills/torchforge-rl-training/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "torchforge-rl-training", then confirm the skill loads.Cursor copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gemini skills install https://github.com/Orchestra-Research/AI-Research-SKILLs.git --path 06-post-training/torchforge--scope user (default) or --scope workspace; --path is the subfolder of the repo that holds the skill; --consent skips the security confirmation prompt.
$ npx skills add Orchestra-Research/AI-Research-SKILLs --skill torchforge-rl-training -a gemini-cliProject install goes to .agents/skills/; add -g for ~/.gemini/skills/.
$ gh skill install Orchestra-Research/AI-Research-SKILLs torchforge-rl-training --agent gemini-cliProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/Orchestra-Research/AI-Research-SKILLs.git skills-src && mkdir -p .gemini/skills && cp -r skills-src/06-post-training/torchforge .gemini/skills/torchforge-rl-training && rm -rf skills-srcUse ~/.gemini/skills/ instead of .gemini/skills for a personal install, then run /skills reload.
Gemini CLI skills documentation · loads skills from .gemini/skills/, .agents/skills/
Install the "torchforge-rl-training" agent skill from https://github.com/Orchestra-Research/AI-Research-SKILLs/tree/main/06-post-training/torchforge into .gemini/skills/torchforge-rl-training/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "torchforge-rl-training", then confirm the skill loads.Gemini CLI copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gh skill install Orchestra-Research/AI-Research-SKILLs torchforge-rl-trainingInstalls for Copilot at project scope by default; add --scope user for a personal install. Preview a skill first with gh skill preview. Needs GitHub CLI 2.90.0 or later (public preview).
$ npx skills add Orchestra-Research/AI-Research-SKILLs --skill torchforge-rl-training -a github-copilotProject install goes to .agents/skills/; add -g for ~/.copilot/skills/.
$ git clone --depth 1 https://github.com/Orchestra-Research/AI-Research-SKILLs.git skills-src && mkdir -p .github/skills && cp -r skills-src/06-post-training/torchforge .github/skills/torchforge-rl-training && rm -rf skills-srcUse ~/.copilot/skills/ instead of .github/skills for a personal install. Commit .github/skills so cloud agent and code review can use it.
GitHub Copilot skills documentation · loads skills from .github/skills/, .claude/skills/, .agents/skills/
Install the "torchforge-rl-training" agent skill from https://github.com/Orchestra-Research/AI-Research-SKILLs/tree/main/06-post-training/torchforge into .github/skills/torchforge-rl-training/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "torchforge-rl-training", then confirm the skill loads.GitHub Copilot copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add Orchestra-Research/AI-Research-SKILLs --skill torchforge-rl-training -a opencodeOpenCode documents no install command of its own. Project install goes to .agents/skills/; add -g for ~/.config/opencode/skills/.
$ gh skill install Orchestra-Research/AI-Research-SKILLs torchforge-rl-training --agent opencodeProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/Orchestra-Research/AI-Research-SKILLs.git skills-src && mkdir -p .opencode/skills && cp -r skills-src/06-post-training/torchforge .opencode/skills/torchforge-rl-training && rm -rf skills-srcUse ~/.config/opencode/skills/ instead of .opencode/skills for a personal install.
OpenCode skills documentation · loads skills from .opencode/skills/, .claude/skills/, .agents/skills/
Install the "torchforge-rl-training" agent skill from https://github.com/Orchestra-Research/AI-Research-SKILLs/tree/main/06-post-training/torchforge into .opencode/skills/torchforge-rl-training/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "torchforge-rl-training", then confirm the skill loads.OpenCode copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
torchforge-rl-trainingGuides reinforcement-learning research with torchforge, Meta's PyTorch-native library that keeps RL algorithms apart from infrastructure, including GRPO math-reasoning runs.
torchforge splits reinforcement learning into algorithm code that you write and infrastructure it handles for you: distributed training, inference and weight sync. It has no Ray dependency, scales through the Monarch actor system, uses TorchTitan for model parallelism, vLLM for inference and TorchStore for syncing, and ships GRPO, DAPO, CISPO, GSPO and SAPO loss functions. The skill covers conda and ROCm installation plus quick-start commands for supervised fine-tuning and GRPO using YAML configs.
Its main workflow trains a math reasoning model with GRPO. The checklist asks for three or more GPUs, split between the trainer, a reference model and the generator, a model from the Hugging Face Hub and a dataset such as GSM8K or MATH. Steps then cover writing the config and defining rewards with the MathReward and ThinkingReward helpers. The skill warns that torchforge is experimental and directs you to miles or verl for stability. The excerpt is cut off at the launch step.
6 steps, taken from the step headings in SKILL.md.
Read from SKILL.md and the folder at commit 773a529. It shows what the files ask for, not the result of running them.
Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.
From allowed-tools in the SKILL.md frontmatter.
Shell commands in SKILL.md call:
pythoncondaFrom the folder's file list and the shell code blocks in SKILL.md.
Links to these hosts (documentation or services it may open):
github.commeta-pytorch.orgdiscord.ggFrom URLs in SKILL.md, links to its own repository left out.
Names no API keys, tokens, secrets or passwords.
From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.
torchforge RL Training loads about 2.5k tokens when it runs, and up to ~6.3k if it reads all its reference files. Until then it costs about 65 tokens; SKILL.md has 368 words of instructions outside code blocks.
Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.
The automated check found no risky patterns in SKILL.md.
Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.
The full file from Orchestra-Research/AI-Research-SKILLs at commit 773a529, republished under its MIT licence (© Orchestra-Research). 368 words, ~2,489 tokens.
.claude/skills/torchforge-rl-training/SKILL.md (or your agent's skills folder). This skill also uses 2 other files; get the full folder from GitHub.torchforge is Meta's PyTorch-native RL library that separates infrastructure concerns from algorithm concerns. It enables rapid RL research by letting you focus on algorithms while handling distributed training, inference, and weight sync automatically.
Choose torchforge when you need:
Consider alternatives when:
┌─────────────────────────────────────────────────────────┐
│ Application Layer (Your Code) │
│ - Define reward models, loss functions, sampling │
└─────────────────────┬───────────────────────────────────┘
│
┌─────────────────────▼───────────────────────────────────┐
│ Forge API Layer │
│ - Episode, Group dataclasses │
│ - Service interfaces (async/await) │
└─────────────────────┬───────────────────────────────────┘
│
┌─────────────────────▼───────────────────────────────────┐
│ Distributed Services (Monarch) │
│ ├── Trainer (TorchTitan FSDP) │
│ ├── Generator (vLLM inference) │
│ ├── Reference Model (frozen KL baseline) │
│ └── Reward Actors (compute rewards) │
└─────────────────────────────────────────────────────────┘# Create environment
conda create -n forge python=3.12
conda activate forge
# Install (handles PyTorch nightly + dependencies)
./scripts/install.sh
# Verify
python -c "import torch, forge, vllm; print('OK')"./scripts/install_rocm.shpython -m apps.sft.main --config apps/sft/llama3_8b.yamlpython -m apps.grpo.main --config apps/grpo/qwen3_1_7b.yamlUse this workflow for training reasoning models with group-relative advantages.
# config/grpo_math.yaml
model: "Qwen/Qwen2.5-7B-Instruct"
dataset:
path: "openai/gsm8k"
split: "train"
streaming: true
training:
batch_size: 4
learning_rate: 1e-6
seq_len: 4096
dtype: bfloat16
gradient_accumulation_steps: 4
grpo:
n_samples: 8 # Responses per prompt
clip_low: 0.2
clip_high: 0.28
beta: 0.1 # KL penalty coefficient
temperature: 0.7
services:
generator:
procs: 1
num_replicas: 1
with_gpus: true
trainer:
procs: 1
num_replicas: 1
with_gpus: true
ref_model:
procs: 1
num_replicas: 1
with_gpus: true# rewards.py
# Reward functions are in forge.data.rewards
from forge.data.rewards import MathReward, ThinkingReward
import re
# Or define your own reward function
class CustomMathReward:
def __call__(self, prompt: str, response: str, target: str) -> float:
# Extract answer from response
match = re.search(r'\\boxed{([^}]+)}', response)
if not match:
return 0.0
answer = match.group(1).strip()
return 1.0 if answer == target else 0.0python -m apps.grpo.main --config config/grpo_math.yamlUse this workflow to implement new RL algorithms.
# src/forge/losses/custom_loss.py
import torch
import torch.nn as nn
class CustomLoss(nn.Module):
def __init__(self, clip_range: float = 0.2, beta: float = 0.1):
super().__init__()
self.clip_range = clip_range
self.beta = beta
def forward(
self,
logprobs: torch.Tensor,
ref_logprobs: torch.Tensor,
advantages: torch.Tensor,
padding_mask: torch.Tensor,
) -> torch.Tensor:
# Compute importance ratio
ratio = torch.exp(logprobs - ref_logprobs)
# Clipped policy gradient
clipped_ratio = torch.clamp(
ratio,
1 - self.clip_range,
1 + self.clip_range
)
pg_loss = -torch.min(ratio * advantages, clipped_ratio * advantages)
# KL penalty
kl = ref_logprobs - logprobs
# Apply mask and aggregate
masked_loss = (pg_loss + self.beta * kl) * padding_mask
loss = masked_loss.sum() / padding_mask.sum()
return loss# apps/custom/main.py
from forge.losses.custom_loss import CustomLoss
loss_fn = CustomLoss(clip_range=0.2, beta=0.1)
# In training loop
loss = loss_fn(
logprobs=logprobs,
ref_logprobs=ref_logprobs,
advantages=advantages,
padding_mask=padding_mask,
)Use this workflow for scaling to multiple GPUs or nodes.
# config/distributed.yaml
model: "meta-llama/Meta-Llama-3.1-8B-Instruct"
parallelism:
tensor_parallel_degree: 2 # Split model across GPUs
pipeline_parallel_degree: 1
data_parallel_shard_degree: 2
services:
generator:
procs: 2 # 2 processes for TP=2
num_replicas: 1
with_gpus: true
trainer:
procs: 2
num_replicas: 1
with_gpus: true# Submit job
sbatch --nodes=2 --gpus-per-node=8 run_grpo.sh# 8 GPU setup
python -m apps.grpo.main \
--config config/distributed.yaml \
--trainer.procs 4 \
--generator.procs 4torchforge uses dictionary-based batches for training:
# inputs: list of dicts with torch.Tensor values
inputs = [{"tokens": torch.Tensor}]
# targets: list of dicts with training signals
targets = [{
"response": torch.Tensor,
"ref_logprobs": torch.Tensor,
"advantages": torch.Tensor,
"padding_mask": torch.Tensor
}]
# train_step returns loss as float
loss = trainer.train_step(inputs, targets)Generated output from vLLM:
@dataclass
class Completion:
text: str # Generated text
token_ids: list[int] # Token IDs
logprobs: list[float] # Log probabilities
metadata: dict # Custom metadataLoss functions are in the forge.losses module:
from forge.losses import SimpleGRPOLoss, ReinforceLoss
# SimpleGRPOLoss for GRPO training
loss_fn = SimpleGRPOLoss(beta=0.1)
# Forward pass
loss = loss_fn(
logprobs=logprobs,
ref_logprobs=ref_logprobs,
advantages=advantages,
padding_mask=padding_mask
)from forge.losses.reinforce_loss import ReinforceLoss
# With optional importance ratio clipping
loss_fn = ReinforceLoss(clip_ratio=0.2)Symptoms: "Insufficient GPU resources" error
Solutions:
# Reduce service requirements
services:
generator:
procs: 1
with_gpus: true
trainer:
procs: 1
with_gpus: true
# Remove ref_model (uses generator weights)Or use CPU for reference model:
ref_model:
with_gpus: falseSymptoms: CUDA OOM in vLLM
Solutions:
# Reduce batch size
grpo:
n_samples: 4 # Reduce from 8
# Or reduce sequence length
training:
seq_len: 2048Symptoms: Long pauses between training and generation
Solutions:
# Enable RDMA (if available)
export TORCHSTORE_USE_RDMA=1
# Or reduce sync frequency
training:
sync_interval: 10 # Sync every 10 stepsSymptoms: Entropy drops to zero, reward stops improving
Solutions:
# Increase KL penalty
grpo:
beta: 0.2 # Increase from 0.1
# Or add entropy bonus
training:
entropy_coef: 0.01© Orchestra-Research, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file
SKILL.md and 2 other files (references) in 06-post-training/torchforge of Orchestra-Research/AI-Research-SKILLs.
Open the folder on GitHubat commit 773a529
We found 2 copies of this SKILL.md (exact, near-identical or edited) in other folders, from 2 other GitHub owners. This page covers the copy in Orchestra-Research/AI-Research-SKILLs, which our catalogue first saw on October 7, 2026.
torchforge RL Training next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.
| Skill | Stars | Used in | Tokens | Auto-check | Licence | Repo updated |
|---|---|---|---|---|---|---|
| torchforge RL Training this skillOrchestra-Research/AI-Research-SKILLs | 13k | 2 repos | ~2.5k | Automated safety check: Pass | MIT | |
| Magpie Kernel Evaluatoramd/skills | 406 | — | ~2.3k | Automated safety check: Pass | MIT | |
| ML Experiment IterationLeeroo-AI/superml | 195 | — | ~4.8k | Automated safety check: Pass | Apache-2.0 | |
| Alphagenome Predictionsgenomicsxai/alphagenome-pytorch | 162 | — | ~868 | Automated safety check: Pass | Apache-2.0 | |
| ML Training Run VerifierLeeroo-AI/superml | 195 | — | ~3.8k | Automated safety check: Pass | Apache-2.0 | |
| Quark Torch Quant Perfamd/Quark | 181 | — | ~3k | Automated safety check: Pass | MIT |
amd/skills
Benchmarks LLM inference and drives GPU kernel optimization with Magpie.
Leeroo-AI/superml
Produces ranked, evidence-grounded next steps when an ML experiment has stalled, drawing on a Leeroopedia knowledge base or on fetched docs and issues.
genomicsxai/alphagenome-pytorch
Run AlphaGenome-PyTorch to get genomic track predictions — via the agt predict CLI (single locus, BED regions, whole chromosomes, raw FASTA sequences, or per-gene count tables/AnnData), variant…
Leeroo-AI/superml
Checks training code, configs and math against documented framework behavior before an expensive run, citing a knowledge base or official docs for every claim.
amd/Quark
Run, resume, monitor, diagnose, and report Quark Quant-Perf workflows for PyTorch and HuggingFace transformers models.
pytorch/executorch
Export a PyTorch model to .pte format for ExecuTorch. Use when converting models, lowering to edge, or generating .pte files.
Orchestra-Research/AI-Research-SKILLs
Generates music from text descriptions with MusicGen and sound effects with AudioGen, using Meta's AudioCraft PyTorch library with melody and style conditioning.
Orchestra-Research/AI-Research-SKILLs
Runs lm-evaluation-harness to benchmark language models on academic suites such as MMLU, GSM8K and HumanEval, compare models and track training checkpoints.
Orchestra-Research/AI-Research-SKILLs
Guide to using Meta's Segment Anything Model for zero-shot image segmentation with point, box or mask prompts, or automatic mask generation.
Orchestra-Research/AI-Research-SKILLs
Shows how to store documents and embeddings in Chroma, query them by similarity with metadata filters, and persist them to disk for RAG and semantic search projects.
Orchestra-Research/AI-Research-SKILLs
Explains OpenAI's CLIP model for zero-shot image classification, image-text similarity, semantic image search and content moderation, with install steps and code patterns.
Orchestra-Research/AI-Research-SKILLs
Transcribes audio with OpenAI's Whisper: 99 languages, translation to English, language detection, six model sizes and word-level timestamps, from Python or the CLI.
Works with
Categories
Guides reinforcement-learning research with torchforge, Meta's PyTorch-native library that keeps RL algorithms apart from infrastructure, including GRPO math-reasoning runs. torchforge splits reinforcement learning into algorithm code that you write and infrastructure it handles for you: distributed training, inference and weight sync. It has no Ray dependency, scales through the Monarch actor system, uses TorchTitan for model parallelism, vLLM for inference and TorchStore for syncing, and ships GRPO, DAPO, CISPO, GSPO and SAPO loss functions.
torchforge RL Training fits situations like: experimenting with new RL algorithms without rewriting infrastructure; running GRPO training on a math dataset with a custom reward; wanting PyTorch-native RL tooling instead of a Ray-based stack; scaling an RL experiment with Monarch and TorchTitan.
Run `npx skills add Orchestra-Research/AI-Research-SKILLs --skill torchforge-rl-training -a claude-code`. Or copy the skill folder (06-post-training/torchforge in Orchestra-Research/AI-Research-SKILLs) into .claude/skills/torchforge-rl-training in your project. Claude Code loads it when a task matches its description.
Run `npx skills add Orchestra-Research/AI-Research-SKILLs --skill torchforge-rl-training -a codex`. Or copy the skill folder (06-post-training/torchforge in Orchestra-Research/AI-Research-SKILLs) into .agents/skills/torchforge-rl-training in your project. Codex loads it when a task matches its description.
Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add Orchestra-Research/AI-Research-SKILLs --skill torchforge-rl-training -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/torchforge-rl-training, .gemini/skills/torchforge-rl-training, .github/skills/torchforge-rl-training and .opencode/skills/torchforge-rl-training in your project.
Going by SKILL.md and its folder, torchforge RL Training needs the command-line tools its instructions call (python and conda). Our summary lists: A conda environment with torchforge installed; Three or more GPUs for GRPO training; A model from the Hugging Face Hub and a training dataset.
SKILL.md names 3 domains. As links in the text: github.com, meta-pytorch.org and discord.gg. This is read from the text; nothing was executed.
Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.
torchforge RL Training is published under the MIT licence (declared in SKILL.md). It allows redistribution, so the full SKILL.md is shown on this page.
About 2.5k tokens (SKILL.md is roughly 10k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full. Its references folder adds about 3.8k tokens, read only when the agent opens those files.
Skills that share tags, products or a category with torchforge RL Training: Magpie Kernel Evaluator (amd/skills, 406 stars), ML Experiment Iteration (Leeroo-AI/superml, 195 stars), Alphagenome Predictions (genomicsxai/alphagenome-pytorch, 162 stars) and ML Training Run Verifier (Leeroo-AI/superml, 195 stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.
Orchestra-Research (a GitHub organization) maintains it in Orchestra-Research/AI-Research-SKILLs, which has 13,374 GitHub stars. The repository holds 96 skills in this directory. The repository was last updated on June 16, 2026.
Source: Orchestra-Research/AI-Research-SKILLs on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.