New: 98,000+ skills indexed from GitHub repos with 100+ stars · every skill safety-checkedBrowse top skillsTop skills
Esc to closeBrowse all skills
Submit a skillBrowse skills
Submit a skillBrowse skills

Agent skill

Torchrl

by VectorSpaceLab in VectorSpaceLab/AREX-Skill

Use TorchRL for TensorDict-first reinforcement-learning environments, collectors, replay buffers, modules, objectives, LLM/RLHF/VLA workflows, services, rendering, and maintainer-safe repository…

MITAuto-check passedAI & LLM Engineering

Install Torchrl

skills CLI
$ npx skills add VectorSpaceLab/AREX-Skill --skill torchrl -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install VectorSpaceLab/AREX-Skill torchrl --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/VectorSpaceLab/AREX-Skill.git skills-src && mkdir -p .claude/skills && cp -r skills-src/skills/repositories/repo-skills/torchrl .claude/skills/torchrl && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
torchrl
GitHub stars
328
Token cost
~1.5k tokens
SKILL.md length
496 words
Files
8 (incl. scripts, references)
Skills in repo
157
Repo updated
First seen
Licence
MIT

At a glance

Use TorchRL for TensorDict-first reinforcement-learning environments, collectors, replay buffers, modules, objectives, LLM/RLHF/VLA workflows, services, rendering, and maintainer-safe repository…

  • Works in 4 steps: Confirm the installed package and… → For a reusable base smoke, run… → If the task depends on Gym, MuJoCo, DM… → …
  • Tasks that involve Reinforcement learning
  • SKILL.md covers First checks, Route by task, Install and dependency stance and Core mental model, plus 2 more sections
  • Runs Python scripts from its folder; calls python and pip

What it does

Torchrl is an agent skill from VectorSpaceLab/AREX-Skill. Use TorchRL for TensorDict-first reinforcement-learning environments, collectors, replay buffers, modules, objectives, LLM/RLHF/VLA workflows, services, rendering, and maintainer-safe repository changes.

Its SKILL.md is about 1.5k tokens, which your agent loads only when the skill is triggered. The skill folder holds 9 other files, including scripts and reference files (for example `references/backend-compatibility.md`, `references/install-and-extras.md` and `references/repo-provenance.md`).

It sits in AI & LLM Engineering, covering Reinforcement learning and Fine-tuning. The repository describes itself as: A Skill Library for Automated Machine Learning. The licence is MIT.

When your agent uses it

  • Tasks that involve Reinforcement learning
  • Tasks that involve Fine-tuning

Example prompts

  • “/torchrl”

Requirements

  • Python 3

Workflow steps

4 steps, taken from the first numbered list in SKILL.md.

  1. Confirm the installed package and backend scope before making claims
  2. For a reusable base smoke, run scripts/check_torchrl_env.py. It imports the major TorchRL surfaces, runs a native PendulumEnv rollout…
  3. If the task depends on Gym, MuJoCo, DM Control, IsaacLab, VMAS, Ray, vLLM, SGLang, LeRobot/OpenX, video codecs, or CUDA kernels, read…
  4. If you are working in a source checkout, compare it with repository provenance. Refresh this skill if commit, package version, public…

What it can do on your machine

Read from SKILL.md and the folder at commit ac3fe1a. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    Ships 1 file in scripts/ (Python), which the agent can run.

    Shell commands in SKILL.md call:

    • python
    • pip

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    No URLs in SKILL.md. Its commands use pip, which can reach the network depending on how they are called.

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Torchrl loads about 1.5k tokens when it runs, and up to ~5.6k if it reads all its reference files. Until then it costs about 53 tokens; SKILL.md has 496 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~53
When it runs · the whole SKILL.md, loaded when a task matches
~1.5k
With references · SKILL.md plus every file in references/, read only if the agent opens them
~5.6k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); the scripts in this folder are not scanned.

SKILL.md

The full file from VectorSpaceLab/AREX-Skill at commit ac3fe1a, republished under its MIT licence (© VectorSpaceLab). 496 words, ~1,488 tokens.

Download SKILL.mdSave it as .claude/skills/torchrl/SKILL.md (or your agent's skills folder). This skill also uses 7 other files; get the full folder from GitHub.
name
torchrl
description
Use TorchRL for TensorDict-first reinforcement-learning environments, collectors, replay buffers, modules, objectives, LLM/RLHF/VLA workflows, services, rendering, and maintainer-safe repository changes.
disable-model-invocation
true
metadata.disco-role
operating
license
MIT

TorchRL

Use this repo skill when the task involves TorchRL (torchrl), the PyTorch reinforcement-learning library built around TensorDict data, composable environments, collectors, replay buffers, modules, losses, trainers, LLM/RLHF/VLA extensions, services, rendering, or contributing to the pytorch/rl repository.

First checks

  1. Confirm the installed package and backend scope before making claims:

    bash
    python - <<'PY'
import torch, tensordict, torchrl
print('torch', torch.__version__, 'cuda', torch.version.cuda, torch.cuda.is_available())
print('tensordict', tensordict.__version__)
print('torchrl', torchrl.__version__)
PY

  2. For a reusable base smoke, run scripts/check_torchrl_env.py. It imports the major TorchRL surfaces, runs a native PendulumEnv rollout, samples a small replay buffer, inspects rlrender help, and reports optional backend availability without downloading models or starting services.

  3. If the task depends on Gym, MuJoCo, DM Control, IsaacLab, VMAS, Ray, vLLM, SGLang, LeRobot/OpenX, video codecs, or CUDA kernels, read backend compatibility and the owning sub-skill's troubleshooting file before deciding whether a CPU result is enough.

  4. If you are working in a source checkout, compare it with repository provenance. Refresh this skill if commit, package version, public entry points, or dirty source state differ materially.

Route by task

Task signalRead
EnvBase, PendulumEnv, GymEnv, specs, TransformedEnv, Compose, transforms, check_env_specs, step_mdp, SerialEnv, ParallelEnv, simulator wrappersenvs-and-transforms
Collector, rollout loops, evaluator, frames_per_batch, sync, backend selection, replay buffers, storages, samplers, prioritized replay, HER, memmap, Ray replaycollectors-and-replay
Actor, ProbabilisticActor, ValueOperator, QValueActor, TensorDictModule keys, specs, distributions, recurrent GRU/LSTM modules, multi-agent models, model-based wrappersmodules-and-policies
PPO/SAC/DQN/DDPG/TD3/IQL/CQL/MAPPO losses, value estimators, set_keys, target updaters, trainers, Hydra configs, SOTA algorithm recipesobjectives-and-training
LLM post-training, RLHF/GRPO/SFT, ChatEnv, LLMCollector, vLLM/SGLang wrappers, VLA schemas/actions, service registry, render CLI, video/checkpoint surfacesllm-vla-and-services
Editing TorchRL source, adding public APIs, tests, docs, benchmarks, deprecations, optional-dep CI labels, GPU markers, Hydra config paritydevelopment-and-testing
Show full SKILL.md (236 more words)Show less

Install and dependency stance

  • General users: pip install torchrl with a PyTorch build appropriate for the task. Match PyTorch and TensorDict versions; TorchRL releases are synchronized with the PyTorch ecosystem.
  • Source contributors: use an editable install only in a checkout, after installing the intended PyTorch build. When using uv with a preselected PyTorch/nightly build, use --no-deps for editable installs to avoid unintended framework downgrades.
  • Install optional extras narrowly. Examples: torchrl[dm_control], torchrl[gym_continuous], torchrl[marl], torchrl[offline-data], torchrl[llm], torchrl[llm-vllm], torchrl[llm-sglang], torchrl[grpo], torchrl[vla], torchrl[rendering], torchrl[video].
  • Do not install broad dev/test/LLM/simulator extras just to answer a CPU-verifiable API question. Document unverified optional backend limits instead.

Read install and extras for the package metadata, console entry points, and safe install/probe commands.

Core mental model

TorchRL components pass structured TensorDict objects through the whole loop:

text
TensorDict -> policy/module writes action/log_prob -> environment writes next/reward/done
           -> collector batches trajectories -> replay buffer stores/samples
           -> loss reads named keys -> optimizer updates ordinary PyTorch parameters

Keep keys explicit, prefer NestedKey tuples for nested data, validate specs early, and route optional backend failures to the narrow owner rather than rewriting the full pipeline.

Tiny CPU integration smoke

For a no-download, no-simulator sanity check across the main RL path, run these bundled helpers from their local skill directories after installing TorchRL:

bash
python scripts/check_torchrl_env.py --steps 3 --check-cli
python sub-skills/envs-and-transforms/scripts/smoke_env_rollout.py --steps 3 --check-specs
python sub-skills/modules-and-policies/scripts/smoke_actor.py
python sub-skills/collectors-and-replay/scripts/smoke_collector.py
python sub-skills/objectives-and-training/scripts/inspect_loss_keys.py --loss ClipPPOLoss

When wiring PPO, remember that ClipPPOLoss defaults sample_log_prob to action_log_prob; make the actor write that key or remap the loss with set_keys(sample_log_prob=...).

Cross-cutting troubleshooting

Read troubleshooting for install/import failures, version mismatches, optional dependency errors, CLI misuse, backend claims, and when to stop instead of silently falling back. Workflow-specific failure matrices live in each sub-skill's references/troubleshooting.md.

© VectorSpaceLab, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

SKILL.md and 7 other files (scripts, references) in skills/repositories/repo-skills/torchrl of VectorSpaceLab/AREX-Skill.

  • SKILL.md
  • references/backend-compatibility.md
  • references/install-and-extras.md
  • references/repo-provenance.md
  • references/repo-routing-metadata.json
  • references/troubleshooting.md
  • scripts/check_torchrl_env.py
  • sub-skills

Open the folder on GitHubat commit ac3fe1a

Compare with similar skills

Torchrl next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Torchrl compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Torchrl this skillVectorSpaceLab/AREX-Skill328—~1.5kAutomated safety check: PassMIT
Hugging Face LLM Trainerhuggingface/skills11k3 repos~7.2kAutomated safety check: PassApache-2.0
Train RlOpenPipe/ART11k—~2.4kAutomated safety check: PassApache-2.0
Qwopus27b Rl TrainingR6410418/Jackrong-llm-finetuning-guide1.7k—~830Automated safety check: PassApache-2.0
Fine Tuning With TrlOrchestra-Research/AI-Research-SKILLs13k7 repos~2.9kAutomated safety check: PassMIT
Optim AgentOptim-Agent/optim-agent801—~1.3kAutomated safety check: PassMIT

Similar skills

  • Hugging Face LLM Trainer

    huggingface/skills

    Official

    Trains or fine-tunes language and vision models with TRL or Unsloth on Hugging Face Jobs cloud GPUs, then converts the results to GGUF.

    11k GitHub starsUsed in 3 repos~7.2k tokens
    AI & LLM EngineeringAuto-check passed

  • Train Rl

    OpenPipe/ART

    RL training reference for the ART framework. An agent skill from OpenPipe/ART.

    11k GitHub stars~2.4k tokensUpdated today
    AI & LLM EngineeringAuto-check passed

  • Qwopus27b Rl Training

    R6410418/Jackrong-llm-finetuning-guide

    Prepare, validate, launch-plan, monitor, resume, and stop configurable Qwopus 27B reinforcement-learning workflows for GRPO or GSPO.

    1.7k GitHub stars~830 tokensUpdated 2 mo ago
    AI & LLM EngineeringAuto-check passed

  • Fine Tuning With Trl

    Orchestra-Research/AI-Research-SKILLs

    Fine-tune LLMs using reinforcement learning with TRL - SFT for instruction tuning, DPO for preference alignment, PPO/GRPO for reward optimization, and reward model training.

    13k GitHub starsUsed in 7 repos~2.9k tokens
    AI & LLM EngineeringAuto-check passed

  • Optim Agent

    Optim-Agent/optim-agent

    A skill your agent uses when the user wants to optimize configurable system parameters against a measurable scalar objective, especially for model training, inference, quantitative strategies…

    801 GitHub stars~1.3k tokensUpdated 1 mo ago
    AI & LLM EngineeringAuto-check passed

  • Safactory Workflows

    AI45Lab/SAfactory

    Integrate a benchmark or custom environment into SAfactory using fixed adapter templates and local contract tests, optionally run Docker/RJob evaluation, or prepare GRPO/RL training.

    236 GitHub stars~1.8k tokensUpdated 14 days ago
    AI & LLM EngineeringAuto-check passed

More from VectorSpaceLab/AREX-Skill

All 157 skills in this repo

  • Agent Lightning

    VectorSpaceLab/AREX-Skill

    Use this repo skill for Agent Lightning package tasks: authoring trainable agents, tracing rewards and spans, running LightningStore/Trainer loops, using agl CLI services, choosing examples, and…

    328 GitHub stars~1.4k tokensUpdated 1 mo ago
    Auto-check passed

  • Agent Tools

    VectorSpaceLab/AREX-Skill

    A skill your agent uses when configuring LiteLLM for MCP tools, A2A agents, Claude Code/Cursor agent gateway traffic, MCP auth/OAuth, tool permissions, semantic filtering, or agent-specific proxy…

    328 GitHub stars~1.2k tokensUpdated 1 mo ago
    Auto-check passed

  • Agents And Awel

    VectorSpaceLab/AREX-Skill

    Build and debug DB-GPT agents, tools, skills, teams, and AWEL workflows, including deterministic local DAG runs and HTTP-trigger topology without assuming an LLM, credential, or external service.

    328 GitHub stars~2.3k tokensUpdated 1 mo ago
    Auto-check passed

  • Agents And Middleware

    VectorSpaceLab/AREX-Skill

    Work on the actively maintained LangChain v1 agent package: initchatmodel, createagent, structured output, tools, middleware, embeddings initialization, provider routing, and agent runtime…

    328 GitHub stars~1.2k tokensUpdated 1 mo ago
    Auto-check passed

  • Agents Workflows

    VectorSpaceLab/AREX-Skill

    A skill your agent uses for giskard.agents async chat workflows, tools, prompt templates, structured outputs, retries, rate limiting, embeddings, and optional LiteLLM backend.

    328 GitHub stars~500 tokensUpdated 1 mo ago
    Auto-check passed

  • Alphafold3

    VectorSpaceLab/AREX-Skill

    A skill your agent uses for AlphaFold 3 input preparation, prediction command planning, output interpretation, and Python API inspection.

    328 GitHub stars~1.2k tokensUpdated 1 mo ago
    Auto-check passed

Categories

Questions about Torchrl

What does Torchrl do?

Use TorchRL for TensorDict-first reinforcement-learning environments, collectors, replay buffers, modules, objectives, LLM/RLHF/VLA workflows, services, rendering, and maintainer-safe repository…. Torchrl is an agent skill from VectorSpaceLab/AREX-Skill. Use TorchRL for TensorDict-first reinforcement-learning environments, collectors, replay buffers, modules, objectives, LLM/RLHF/VLA workflows, services, rendering, and maintainer-safe repository changes.

When should I use Torchrl?

Torchrl fits situations like: tasks that involve Reinforcement learning; tasks that involve Fine-tuning.

How do I install Torchrl in Claude Code?

Run `npx skills add VectorSpaceLab/AREX-Skill --skill torchrl -a claude-code`. Or copy the skill folder (skills/repositories/repo-skills/torchrl in VectorSpaceLab/AREX-Skill) into .claude/skills/torchrl in your project. Claude Code loads it when a task matches its description.

How do I install Torchrl in Codex?

Run `npx skills add VectorSpaceLab/AREX-Skill --skill torchrl -a codex`. Or copy the skill folder (skills/repositories/repo-skills/torchrl in VectorSpaceLab/AREX-Skill) into .agents/skills/torchrl in your project. Codex loads it when a task matches its description.

Can I use Torchrl in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add VectorSpaceLab/AREX-Skill --skill torchrl -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/torchrl, .gemini/skills/torchrl, .github/skills/torchrl and .opencode/skills/torchrl in your project.

What does Torchrl need to run?

Going by SKILL.md and its folder, Torchrl needs Python for the scripts in its folder and the command-line tools its instructions call (python and pip). Our summary lists: Python 3.

Does Torchrl access the network?

SKILL.md contains no URLs. Its commands use pip, which can reach the network depending on how they are called. This is read from the text; nothing was executed.

Is Torchrl safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. The check reads SKILL.md only: the scripts in the folder are not scanned, so read them before running anything.

What licence does Torchrl use?

Torchrl is published under the MIT licence (declared in SKILL.md). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Torchrl use?

About 1.5k tokens (SKILL.md is roughly 6k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full. Its references folder adds about 4.1k tokens, read only when the agent opens those files.

What are the alternatives to Torchrl?

Skills that share tags, products or a category with Torchrl: Hugging Face LLM Trainer (huggingface/skills, 11k stars), Train Rl (OpenPipe/ART, 11k stars), Qwopus27b Rl Training (R6410418/Jackrong-llm-finetuning-guide, 1.7k stars) and Fine Tuning With Trl (Orchestra-Research/AI-Research-SKILLs, 13k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Torchrl?

VectorSpaceLab (a GitHub organization) maintains it in VectorSpaceLab/AREX-Skill, which has 328 GitHub stars. The repository holds 157 skills in this directory. The repository was last updated on September 3, 2026.

Source: VectorSpaceLab/AREX-Skill on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.