Peft Fine Tuning
Orchestra-Research/AI-Research-SKILLs
Parameter-efficient fine-tuning for LLMs using LoRA, QLoRA, and 25+ methods.
Use torchtune model/tokenizer builders, PEFT modules, losses, conversion utilities, and modeling components safely.
Install Models And Modules
$ npx skills add VectorSpaceLab/AREX-Skill --skill models-and-modules -a claude-codeProject install by default; add -g for ~/.claude/skills/.
$ gh skill install VectorSpaceLab/AREX-Skill models-and-modules --agent claude-codeProject scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).
$ git clone --depth 1 https://github.com/VectorSpaceLab/AREX-Skill.git skills-src && mkdir -p .claude/skills && cp -r skills-src/skills/repositories/repo-skills/torchtune/sub-skills/models-and-modules .claude/skills/models-and-modules && rm -rf skills-srcUse ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.
Claude Code skills documentation · loads skills from .claude/skills/
Install the "models-and-modules" agent skill from https://github.com/VectorSpaceLab/AREX-Skill/tree/main/skills/repositories/repo-skills/torchtune/sub-skills/models-and-modules into .claude/skills/models-and-modules/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "models-and-modules", then confirm the skill loads.Claude Code copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$skill-installer install https://github.com/VectorSpaceLab/AREX-Skill/tree/main/skills/repositories/repo-skills/torchtune/sub-skills/models-and-modulesType this inside Codex. $skill-installer <name> installs a curated skill from openai/skills. The installer writes to $CODEX_HOME/skills (default ~/.codex/skills). Restart Codex if the skill does not show up.
$ npx skills add VectorSpaceLab/AREX-Skill --skill models-and-modules -a codexProject install goes to .agents/skills/; add -g for ~/.codex/skills/.
$ gh skill install VectorSpaceLab/AREX-Skill models-and-modules --agent codexProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/VectorSpaceLab/AREX-Skill.git skills-src && mkdir -p .agents/skills && cp -r skills-src/skills/repositories/repo-skills/torchtune/sub-skills/models-and-modules .agents/skills/models-and-modules && rm -rf skills-srcUse ~/.agents/skills/ instead of .agents/skills for a personal install.
Codex skills documentation · loads skills from .agents/skills/
Install the "models-and-modules" agent skill from https://github.com/VectorSpaceLab/AREX-Skill/tree/main/skills/repositories/repo-skills/torchtune/sub-skills/models-and-modules into .agents/skills/models-and-modules/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "models-and-modules", then confirm the skill loads.Codex copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add VectorSpaceLab/AREX-Skill --skill models-and-modules -a cursorProject install goes to .agents/skills/; add -g for ~/.cursor/skills/.
$ gh skill install VectorSpaceLab/AREX-Skill models-and-modules --agent cursorProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/VectorSpaceLab/AREX-Skill.git skills-src && mkdir -p .cursor/skills && cp -r skills-src/skills/repositories/repo-skills/torchtune/sub-skills/models-and-modules .cursor/skills/models-and-modules && rm -rf skills-srcUse ~/.cursor/skills/ instead of .cursor/skills for a personal install.
Cursor skills documentation · loads skills from .cursor/skills/, .agents/skills/, .claude/skills/, .codex/skills/
Install the "models-and-modules" agent skill from https://github.com/VectorSpaceLab/AREX-Skill/tree/main/skills/repositories/repo-skills/torchtune/sub-skills/models-and-modules into .cursor/skills/models-and-modules/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "models-and-modules", then confirm the skill loads.Cursor copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gemini skills install https://github.com/VectorSpaceLab/AREX-Skill.git --path skills/repositories/repo-skills/torchtune/sub-skills/models-and-modules--scope user (default) or --scope workspace; --path is the subfolder of the repo that holds the skill; --consent skips the security confirmation prompt.
$ npx skills add VectorSpaceLab/AREX-Skill --skill models-and-modules -a gemini-cliProject install goes to .agents/skills/; add -g for ~/.gemini/skills/.
$ gh skill install VectorSpaceLab/AREX-Skill models-and-modules --agent gemini-cliProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/VectorSpaceLab/AREX-Skill.git skills-src && mkdir -p .gemini/skills && cp -r skills-src/skills/repositories/repo-skills/torchtune/sub-skills/models-and-modules .gemini/skills/models-and-modules && rm -rf skills-srcUse ~/.gemini/skills/ instead of .gemini/skills for a personal install, then run /skills reload.
Gemini CLI skills documentation · loads skills from .gemini/skills/, .agents/skills/
Install the "models-and-modules" agent skill from https://github.com/VectorSpaceLab/AREX-Skill/tree/main/skills/repositories/repo-skills/torchtune/sub-skills/models-and-modules into .gemini/skills/models-and-modules/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "models-and-modules", then confirm the skill loads.Gemini CLI copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gh skill install VectorSpaceLab/AREX-Skill models-and-modulesInstalls for Copilot at project scope by default; add --scope user for a personal install. Preview a skill first with gh skill preview. Needs GitHub CLI 2.90.0 or later (public preview).
$ npx skills add VectorSpaceLab/AREX-Skill --skill models-and-modules -a github-copilotProject install goes to .agents/skills/; add -g for ~/.copilot/skills/.
$ git clone --depth 1 https://github.com/VectorSpaceLab/AREX-Skill.git skills-src && mkdir -p .github/skills && cp -r skills-src/skills/repositories/repo-skills/torchtune/sub-skills/models-and-modules .github/skills/models-and-modules && rm -rf skills-srcUse ~/.copilot/skills/ instead of .github/skills for a personal install. Commit .github/skills so cloud agent and code review can use it.
GitHub Copilot skills documentation · loads skills from .github/skills/, .claude/skills/, .agents/skills/
Install the "models-and-modules" agent skill from https://github.com/VectorSpaceLab/AREX-Skill/tree/main/skills/repositories/repo-skills/torchtune/sub-skills/models-and-modules into .github/skills/models-and-modules/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "models-and-modules", then confirm the skill loads.GitHub Copilot copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add VectorSpaceLab/AREX-Skill --skill models-and-modules -a opencodeOpenCode documents no install command of its own. Project install goes to .agents/skills/; add -g for ~/.config/opencode/skills/.
$ gh skill install VectorSpaceLab/AREX-Skill models-and-modules --agent opencodeProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/VectorSpaceLab/AREX-Skill.git skills-src && mkdir -p .opencode/skills && cp -r skills-src/skills/repositories/repo-skills/torchtune/sub-skills/models-and-modules .opencode/skills/models-and-modules && rm -rf skills-srcUse ~/.config/opencode/skills/ instead of .opencode/skills for a personal install.
OpenCode skills documentation · loads skills from .opencode/skills/, .claude/skills/, .agents/skills/
Install the "models-and-modules" agent skill from https://github.com/VectorSpaceLab/AREX-Skill/tree/main/skills/repositories/repo-skills/torchtune/sub-skills/models-and-modules into .opencode/skills/models-and-modules/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "models-and-modules", then confirm the skill loads.OpenCode copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
Facts
- Skill name
models-and-modules- GitHub stars
- 328
- Token cost
- ~1.2k tokens
- SKILL.md length
- 467 words
- Files
- 6 (incl. scripts, references)
- Skills in repo
- 157
- Repo updated
- First seen
- Licence
- BSD-3-Clause
At a glance
Use torchtune model/tokenizer builders, PEFT modules, losses, conversion utilities, and modeling components safely.
- Works in 6 steps: Identify the model family and checkpoint… → Match the tokenizer or model transform… → For adapter work, choose a family lora_*… → …
- Tasks that involve Fine-tuning
- SKILL.md covers Route Here For, Standard Workflow, Bundled Helper and Read Next, plus 1 more section
- Runs Python scripts from its folder; calls python
What it does
Models And Modules is an agent skill from VectorSpaceLab/AREX-Skill. Use torchtune model/tokenizer builders, PEFT modules, losses, conversion utilities, and modeling components safely.
Its SKILL.md is about 1.2k tokens, which your agent loads only when the skill is triggered. The skill folder holds 7 other files, including scripts and reference files (for example `references/model-catalog.md`, `references/module-api-reference.md` and `references/peft-and-adapters.md`).
It sits in AI & LLM Engineering, covering Fine-tuning. The repository describes itself as: A Skill Library for Automated Machine Learning. The licence is BSD-3-Clause.
When your agent uses it
- Tasks that involve Fine-tuning
Example prompts
- “/models-and-modules”
Requirements
- Python 3
Workflow steps
6 steps, taken from the first numbered list in SKILL.md.
- Identify the model family and checkpoint provenance before choosing builder names; use model catalog for public dotpaths.
- Match the tokenizer or model transform to the same family and downloaded artifact layout before training, generation, or dataset transforms.
- For adapter work, choose a family lora_* or qlora_* builder first, then use PEFT and adapters for lora_attn_modules, trainable params, and…
- For custom code, import public modules from torchtune.models, torchtune.modules, torchtune.modules.peft, torchtune.modules.loss, or…
- Use module API reference for component signatures, losses, generation helpers, MoE, low precision, and conversion utilities.
- Diagnose optional dependency, tokenizer, gated checkpoint, LoRA target, QLoRA, and state-dict issues with troubleshooting before launching…
What it can do on your machine
Read from SKILL.md and the folder at commit ac3fe1a. It shows what the files ask for, not the result of running them.
Tool permissions
Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.
From allowed-tools in the SKILL.md frontmatter.
Runs code
Ships 1 file in scripts/ (Python), which the agent can run.
Shell commands in SKILL.md call:
python
From the folder's file list and the shell code blocks in SKILL.md.
Network
No URLs in SKILL.md.
From URLs in SKILL.md, links to its own repository left out.
Credentials
Names no API keys, tokens, secrets or passwords.
From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.
Context cost
Models And Modules loads about 1.2k tokens when it runs, and up to ~8.8k if it reads all its reference files. Until then it costs about 34 tokens; SKILL.md has 467 words of instructions outside code blocks.
Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.
Safety
Auto-check passed
The automated check found no risky patterns in SKILL.md.
Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); the scripts in this folder are not scanned.
SKILL.md
The full file from VectorSpaceLab/AREX-Skill at commit ac3fe1a, republished under its BSD-3-Clause licence (© VectorSpaceLab). 467 words, ~1,188 tokens.
Download SKILL.mdSave it as
.claude/skills/models-and-modules/SKILL.md (or your agent's skills folder). This skill also uses 5 other files; get the full folder from GitHub.- name
- models-and-modules
- description
- Use torchtune model/tokenizer builders, PEFT modules, losses, conversion utilities, and modeling components safely.
- disable-model-invocation
- true
- metadata.disco-role
- operating
- license
- BSD 3-Clause
models-and-modules
Use this sub-skill when a torchtune task is about model-family builders, tokenizer/model-transform builders, LoRA/QLoRA/DoRA adapters, transformer/attention/loss modules, MoE and low-precision pieces, state-dict conversion, or custom model components.
Route Here For
- Choosing public
_component_dotpaths undertorchtune.models.*ortorchtune.modules.*for YAML configs or custom code. - Pairing model builders with matching tokenizer or multimodal transform builders before dataset or recipe wiring.
- Adding LoRA, QLoRA, DoRA, QAT-LoRA, adapter-state extraction, adapter merging, or trainable-parameter setup.
- Using torchtune modeling blocks such as attention, decoder layers, KV cache, RoPE, RMSNorm, chunked/KL losses, MoE experts, and NF4 linear layers.
- Converting state-dict key formats between torchtune, Meta, Hugging Face, and PEFT adapter conventions.
- Writing custom config-friendly builders without importing the intentionally non-importable
recipespackage.
Standard Workflow
- Identify the model family and checkpoint provenance before choosing builder names; use model catalog for public dotpaths.
- Match the tokenizer or model transform to the same family and downloaded artifact layout before training, generation, or dataset transforms.
- For adapter work, choose a family
lora_*orqlora_*builder first, then use PEFT and adapters forlora_attn_modules, trainable params, and merge/export behavior. - For custom code, import public modules from
torchtune.models,torchtune.modules,torchtune.modules.peft,torchtune.modules.loss, ortorchtune.generation; avoid private underscore dotpaths in configs. - Use module API reference for component signatures, losses, generation helpers, MoE, low precision, and conversion utilities.
- Diagnose optional dependency, tokenizer, gated checkpoint, LoRA target, QLoRA, and state-dict issues with troubleshooting before launching expensive jobs.
Bundled Helper
Run the helper from this sub-skill directory to inspect callable public model exports in the active environment without instantiating large models:
python scripts/inspect_model_builders.py --families llama3 qwen2_5 --format tableIt imports family modules, lists callable public exports and signatures, and reports optional-dependency import failures instead of downloading checkpoints or constructing models.
Show full SKILL.md (184 more words)Show less
Read Next
- Model catalog for family builders, tokenizer notes, multimodal transforms, and public dotpath examples.
- Module API reference for attention/decoder blocks, losses, generation, MoE, low precision, export variants, and conversion functions.
- PEFT and adapters for LoRA/QLoRA/DoRA/QAT-LoRA config patterns and adapter state dict guidance.
- Troubleshooting for private dotpaths, gated downloads, tokenizer mismatches, adapter target names, QLoRA dependencies, conversion failures, and export variants.
- data-and-datasets for dataset transforms, message schemas, packing, and collators.
- post-training-recipes for recipe selection and training launch planning.
- inference-evaluation-quantization for generation workflows, evaluation, and post-training quantization routing.
- cli-and-config for
tune cp,tune cat,tune validate,_component_, and override mechanics.
Guardrails
- Use public dotpaths such as
torchtune.models.llama3.llama3_8b, not private implementation paths liketorchtune.models.llama3._model_builders.llama3_8b. - Do not
import recipes; usetune run, registry names, copied config files, or custom recipe files launched through the CLI. - Do not instantiate full model builders just to inspect names; use signatures, configs, or the bundled inspection helper.
- Do not leak checkpoint locations, token files, credentials, local environments, or machine-specific paths into reusable configs or skill content.
- Keep dataset schemas in
data-and-datasets, recipe launch construction inpost-training-recipes, and generation/evaluation/quantization workflows ininference-evaluation-quantization.
© VectorSpaceLab, BSD-3-Clause. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file
Files
SKILL.md and 5 other files (scripts, references) in skills/repositories/repo-skills/torchtune/sub-skills/models-and-modules of VectorSpaceLab/AREX-Skill.
- SKILL.md
- references/model-catalog.md
- references/module-api-reference.md
- references/peft-and-adapters.md
- references/troubleshooting.md
- scripts/inspect_model_builders.py
Open the folder on GitHubat commit ac3fe1a
Compare with similar skills
Models And Modules next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.
| Skill | Stars | Used in | Tokens | Auto-check | Licence | Repo updated |
|---|---|---|---|---|---|---|
| Models And Modules this skillVectorSpaceLab/AREX-Skill | 328 | — | ~1.2k | Automated safety check: Pass | BSD-3-Clause | |
| Peft Fine TuningOrchestra-Research/AI-Research-SKILLs | 13k | 9 repos | ~3.1k | Automated safety check: Pass | MIT | |
| Hugging Face LLM Trainerhuggingface/skills | 11k | 3 repos | ~7.2k | Automated safety check: Pass | Apache-2.0 | |
| Sentence-Transformers Training Routerhuggingface/skills | 11k | 1 repos | ~2.6k | Automated safety check: Pass | Apache-2.0 | |
| Dataset Evaluationawslabs/agent-plugins | 915 | 2 repos | ~1.3k | Automated safety check: Pass | Apache-2.0 | |
| Train RlOpenPipe/ART | 11k | — | ~2.4k | Automated safety check: Pass | Apache-2.0 |
Similar skills
- Official
Hugging Face LLM Trainer
huggingface/skills
Trains or fine-tunes language and vision models with TRL or Unsloth on Hugging Face Jobs cloud GPUs, then converts the results to GGUF.
- Official
Sentence-Transformers Training Router
huggingface/skills
Routes a sentence-transformers training task to the right model type and required reference docs and example scripts, covering bi-encoders, rerankers, sparse and multi-vector models.
- Official
Dataset Evaluation
awslabs/agent-plugins
Validates dataset formatting and quality for SageMaker model fine-tuning (SFT, DPO, or RLVR).
Train Rl
OpenPipe/ART
RL training reference for the ART framework. An agent skill from OpenPipe/ART.
Qwopus27b Rl Training
R6410418/Jackrong-llm-finetuning-guide
Prepare, validate, launch-plan, monitor, resume, and stop configurable Qwopus 27B reinforcement-learning workflows for GRPO or GSPO.
More from VectorSpaceLab/AREX-Skill
All 157 skills in this repoAgent Lightning
VectorSpaceLab/AREX-Skill
Use this repo skill for Agent Lightning package tasks: authoring trainable agents, tracing rewards and spans, running LightningStore/Trainer loops, using agl CLI services, choosing examples, and…
Agent Tools
VectorSpaceLab/AREX-Skill
A skill your agent uses when configuring LiteLLM for MCP tools, A2A agents, Claude Code/Cursor agent gateway traffic, MCP auth/OAuth, tool permissions, semantic filtering, or agent-specific proxy…
Agents And Awel
VectorSpaceLab/AREX-Skill
Build and debug DB-GPT agents, tools, skills, teams, and AWEL workflows, including deterministic local DAG runs and HTTP-trigger topology without assuming an LLM, credential, or external service.
Agents And Middleware
VectorSpaceLab/AREX-Skill
Work on the actively maintained LangChain v1 agent package: initchatmodel, createagent, structured output, tools, middleware, embeddings initialization, provider routing, and agent runtime…
Agents Workflows
VectorSpaceLab/AREX-Skill
A skill your agent uses for giskard.agents async chat workflows, tools, prompt templates, structured outputs, retries, rate limiting, embeddings, and optional LiteLLM backend.
Alphafold3
VectorSpaceLab/AREX-Skill
A skill your agent uses for AlphaFold 3 input preparation, prediction command planning, output interpretation, and Python API inspection.
Related
Categories
Questions about Models And Modules
What does Models And Modules do?
Use torchtune model/tokenizer builders, PEFT modules, losses, conversion utilities, and modeling components safely. Models And Modules is an agent skill from VectorSpaceLab/AREX-Skill. Use torchtune model/tokenizer builders, PEFT modules, losses, conversion utilities, and modeling components safely.
When should I use Models And Modules?
Models And Modules fits situations like: tasks that involve Fine-tuning.
How do I install Models And Modules in Claude Code?
Run `npx skills add VectorSpaceLab/AREX-Skill --skill models-and-modules -a claude-code`. Or copy the skill folder (skills/repositories/repo-skills/torchtune/sub-skills/models-and-modules in VectorSpaceLab/AREX-Skill) into .claude/skills/models-and-modules in your project. Claude Code loads it when a task matches its description.
How do I install Models And Modules in Codex?
Run `npx skills add VectorSpaceLab/AREX-Skill --skill models-and-modules -a codex`. Or copy the skill folder (skills/repositories/repo-skills/torchtune/sub-skills/models-and-modules in VectorSpaceLab/AREX-Skill) into .agents/skills/models-and-modules in your project. Codex loads it when a task matches its description.
Can I use Models And Modules in Cursor, Gemini CLI or GitHub Copilot?
Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add VectorSpaceLab/AREX-Skill --skill models-and-modules -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/models-and-modules, .gemini/skills/models-and-modules, .github/skills/models-and-modules and .opencode/skills/models-and-modules in your project.
What does Models And Modules need to run?
Going by SKILL.md and its folder, Models And Modules needs Python for the scripts in its folder and the command-line tools its instructions call (python). Our summary lists: Python 3.
Does Models And Modules access the network?
SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.
Is Models And Modules safe to install?
Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. The check reads SKILL.md only: the scripts in the folder are not scanned, so read them before running anything.
What licence does Models And Modules use?
Models And Modules is published under the BSD-3-Clause licence (declared in SKILL.md). It allows redistribution, so the full SKILL.md is shown on this page.
How many tokens does Models And Modules use?
About 1.2k tokens (SKILL.md is roughly 4.8k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full. Its references folder adds about 7.6k tokens, read only when the agent opens those files.
What are the alternatives to Models And Modules?
Skills that share tags, products or a category with Models And Modules: Peft Fine Tuning (Orchestra-Research/AI-Research-SKILLs, 13k stars), Hugging Face LLM Trainer (huggingface/skills, 11k stars), Sentence-Transformers Training Router (huggingface/skills, 11k stars) and Dataset Evaluation (awslabs/agent-plugins, 915 stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.
Who maintains Models And Modules?
VectorSpaceLab (a GitHub organization) maintains it in VectorSpaceLab/AREX-Skill, which has 328 GitHub stars. The repository holds 157 skills in this directory. The repository was last updated on September 3, 2026.
Source: VectorSpaceLab/AREX-Skill on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.