New: 98,000+ skills indexed from GitHub repos with 100+ stars · every skill safety-checkedBrowse top skillsTop skills
Esc to closeBrowse all skills
Submit a skillBrowse skills
Submit a skillBrowse skills

Agent skill

Axolotl Fine-Tuning Reference

by Orchestra-Research in Orchestra-Research/AI-Research-SKILLs

Quick-reference help for fine-tuning language models with Axolotl, covering YAML configs, FSDP, context parallelism, compressed saves and dataset formats.

MITAuto-check passedAI & LLM Engineering

Install Axolotl Fine-Tuning Reference

skills CLI
$ npx skills add Orchestra-Research/AI-Research-SKILLs --skill axolotl -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install Orchestra-Research/AI-Research-SKILLs axolotl --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/Orchestra-Research/AI-Research-SKILLs.git skills-src && mkdir -p .claude/skills && cp -r skills-src/03-fine-tuning/axolotl .claude/skills/axolotl && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
axolotl
GitHub stars
13k
Used in
8 other repos
Token cost
~1.2k tokens
SKILL.md length
453 words
Files
5 (incl. references)
Skills in repo
96
Repo updated
First seen
Licence
MIT

At a glance

Quick-reference help for fine-tuning language models with Axolotl, covering YAML configs, FSDP, context parallelism, compressed saves and dataset formats.

  • Works in 2 steps: Re-run the scraper with the same… → The skill will be rebuilt with the…
  • Writing or debugging an Axolotl YAML config for a fine-tuning run
  • SKILL.md covers When to Use This Skill, Quick Reference, Reference Files and Working with This Skill, plus 3 more sections
  • Instructions only: no scripts, shell commands, URLs or credentials in SKILL.md

What it does

The skill is a reference pack drawn from Axolotl's official documentation. Its quick-reference section collects short patterns: running NCCL tests to check data-transfer speed between GPUs, enabling FSDP in the Axolotl YAML (with `fsdp_version` and `fsdp_config`), choosing a `context_parallel_size` that divides the number of GPUs, and setting `save_compressed: true` to save models in a smaller format that vLLM and llmcompressor can still load, reported as about 40% less disk use.

A worked example explains the batch-size effect of context parallelism: with 8 GPUs and a size of 4, only 2 distinct batches run per step and the global batch drops from 16 to 4. Other notes cover writing a custom integration in any installed Python package and handling single and batched examples when dropping long sequences. Code examples touch the Modal cloud runner and the trainer class, and the `references` folder has api, dataset-formats, index and other pages. The description mentions LoRA, QLoRA, DPO, KTO, ORPO, GRPO and multimodal support.

When your agent uses it

  • Writing or debugging an Axolotl YAML config for a fine-tuning run
  • Choosing a dataset format for Axolotl training data
  • Setting up FSDP or context parallelism across several GPUs
  • Looking up Axolotl API classes before extending the trainer

Example prompts

  • “Write an Axolotl config for a QLoRA run on Llama with my dataset in ./data/train.jsonl.”
  • “How does context_parallel_size change my global batch size on 8 GPUs?”
  • “Turn on FSDP in this Axolotl YAML and offload parameters to CPU.”
  • “Which dataset formats does Axolotl accept for chat-style training data?”

Requirements

  • Axolotl installed in a Python environment
  • GPUs for training runs

Workflow steps

2 steps, taken from the first numbered list in SKILL.md.

  1. Re-run the scraper with the same configuration
  2. The skill will be rebuilt with the latest information

What it can do on your machine

Read from SKILL.md and the folder at commit 773a529. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    No scripts in the folder and no shell commands in SKILL.md (its code samples are python).

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    Links to these hosts (documentation or services it may open):

    • github.com

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Axolotl Fine-Tuning Reference loads about 1.2k tokens when it runs, and up to ~78k if it reads all its reference files. Until then it costs about 34 tokens; SKILL.md has 453 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~34
When it runs · the whole SKILL.md, loaded when a task matches
~1.2k
With references · SKILL.md plus every file in references/, read only if the agent opens them
~78k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from Orchestra-Research/AI-Research-SKILLs at commit 773a529, republished under its MIT licence (© Orchestra-Research). 453 words, ~1,197 tokens.

Download SKILL.mdSave it as .claude/skills/axolotl/SKILL.md (or your agent's skills folder). This skill also uses 4 other files; get the full folder from GitHub.
name
axolotl
description
Expert guidance for fine-tuning LLMs with Axolotl - YAML configs, 100+ models, LoRA/QLoRA, DPO/KTO/ORPO/GRPO, multimodal support
version
1.0.0
author
Orchestra Research
license
MIT
tags
Fine-Tuning, Axolotl, LLM, LoRA, QLoRA, DPO, KTO, ORPO, GRPO, YAML, HuggingFace, DeepSpeed, Multimodal
dependencies
axolotl, torch, transformers, datasets, peft, accelerate, deepspeed

Axolotl Skill

Comprehensive assistance with axolotl development, generated from official documentation.

When to Use This Skill

This skill should be triggered when:

  • Working with axolotl
  • Asking about axolotl features or APIs
  • Implementing axolotl solutions
  • Debugging axolotl code
  • Learning axolotl best practices

Quick Reference

Common Patterns

Pattern 1: To validate that acceptable data transfer speeds exist for your training job, running NCCL Tests can help pinpoint bottlenecks, for example:

./build/all_reduce_perf -b 8 -e 128M -f 2 -g 3

Pattern 2: Configure your model to use FSDP in the Axolotl yaml. For example:

fsdp_version: 2
fsdp_config:
  offload_params: true
  state_dict_type: FULL_STATE_DICT
  auto_wrap_policy: TRANSFORMER_BASED_WRAP
  transformer_layer_cls_to_wrap: LlamaDecoderLayer
  reshard_after_forward: true

Pattern 3: The context_parallel_size should be a divisor of the total number of GPUs. For example:

context_parallel_size

Pattern 4: For example: - With 8 GPUs and no sequence parallelism: 8 different batches processed per step - With 8 GPUs and context_parallel_size=4: Only 2 different batches processed per step (each split across 4 GPUs) - If your per-GPU micro_batch_size is 2, the global batch size decreases from 16 to 4

context_parallel_size=4

Pattern 5: Setting save_compressed: true in your configuration enables saving models in a compressed format, which: - Reduces disk space usage by approximately 40% - Maintains compatibility with vLLM for accelerated inference - Maintains compatibility with llmcompressor for further optimization (example: quantization)

save_compressed: true

Pattern 6: Note It is not necessary to place your integration in the integrations folder. It can be in any location, so long as it’s installed in a package in your python env. See this repo for an example: https://github.com/axolotl-ai-cloud/diff-transformer

integrations

Pattern 7: Handle both single-example and batched data. - single example: sample[‘input_ids’] is a list[int] - batched data: sample[‘input_ids’] is a list[list[int]]

utils.trainer.drop_long_seq(sample, sequence_len=2048, min_sequence_len=2)
Example Code Patterns

Example 1 (python):

python
cli.cloud.modal_.ModalCloud(config, app=None)

Example 2 (python):

python
cli.cloud.modal_.run_cmd(cmd, run_folder, volumes=None)

Example 3 (python):

python
core.trainers.base.AxolotlTrainer(
    *_args,
    bench_data_collator=None,
    eval_data_collator=None,
    dataset_tags=None,
    **kwargs,
)

Example 4 (python):

python
core.trainers.base.AxolotlTrainer.log(logs, start_time=None)

Example 5 (python):

python
prompt_strategies.input_output.RawInputOutputPrompter()
Show full SKILL.md (189 more words)Show less

Reference Files

This skill includes comprehensive documentation in references/:

  • api.md - Api documentation
  • dataset-formats.md - Dataset-Formats documentation
  • other.md - Other documentation

Use view to read specific reference files when detailed information is needed.

Working with This Skill

For Beginners

Start with the getting_started or tutorials reference files for foundational concepts.

For Specific Features

Use the appropriate category reference file (api, guides, etc.) for detailed information.

For Code Examples

The quick reference section above contains common patterns extracted from the official docs.

Resources

references/

Organized documentation extracted from official sources. These files contain:

  • Detailed explanations
  • Code examples with language annotations
  • Links to original documentation
  • Table of contents for quick navigation
scripts/

Add helper scripts here for common automation tasks.

assets/

Add templates, boilerplate, or example projects here.

Notes

  • This skill was automatically generated from official documentation
  • Reference files preserve the structure and examples from source docs
  • Code examples include language detection for better syntax highlighting
  • Quick reference patterns are extracted from common usage examples in the docs

Updating

To refresh this skill with updated documentation:

  1. Re-run the scraper with the same configuration
  2. The skill will be rebuilt with the latest information

© Orchestra-Research, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

SKILL.md and 4 other files (references) in 03-fine-tuning/axolotl of Orchestra-Research/AI-Research-SKILLs.

  • SKILL.md
  • references/api.md
  • references/dataset-formats.md
  • references/index.md
  • references/other.md

Open the folder on GitHubat commit 773a529

Used in 8 other repositories

We found 9 copies of this SKILL.md (exact, near-identical or edited) in other folders, from 8 other GitHub owners. This page covers the copy in Orchestra-Research/AI-Research-SKILLs, which our catalogue first saw on October 7, 2026.

Compare with similar skills

Axolotl Fine-Tuning Reference next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Axolotl Fine-Tuning Reference compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Axolotl Fine-Tuning Reference this skillOrchestra-Research/AI-Research-SKILLs13k8 repos~1.2kAutomated safety check: PassMIT
Hugging Face Vision Trainerhuggingface/skills11k1 repos~7.5kAutomated safety check: PassApache-2.0
Aqua Deploymentoracle/accelerated-data-science125—~2.4kAutomated safety check: PassUPL-1.0
LLM Serving Framework BenchmarkBBuf/AI-Infra-Auto-Driven-SKILLS938—~7.5kAutomated safety check: PassNone
Magpie Kernel Evaluatoramd/skills408—~2.3kAutomated safety check: PassMIT
Runpodericrisco/rsc-harness180—~2.8kAutomated safety check: PassMIT

Similar skills

  • Hugging Face Vision Trainer

    huggingface/skills

    Official

    Trains and fine-tunes object detection, image classification and SAM or SAM2 segmentation models on Hugging Face Jobs cloud GPUs and saves the results to the Hub.

    11k GitHub starsUsed in 1 repo~7.5k tokens
    AI & LLM EngineeringAuto-check passed

  • Aqua Deployment

    oracle/accelerated-data-science

    Official

    Deploy LLM models on OCI using AI Quick Actions (AQUA) - single model, multi-model, stacked (LoRA), with GPU shape selection, vLLM configuration, streaming, and tool calling.

    125 GitHub stars~2.4k tokensUpdated 1 mo ago
    AI & LLM EngineeringAuto-check passed

  • LLM Serving Framework Benchmark

    BBuf/AI-Infra-Auto-Driven-SKILLS

    Compares SGLang, vLLM, TensorRT-LLM and TokenSpeed on one model and workload, searching server flags to find the best deployment command within a latency SLA.

    938 GitHub stars~7.5k tokensUpdated 6 days ago
    AI & LLM EngineeringAuto-check passed

  • Magpie Kernel Evaluator

    amd/skills

    Benchmarks LLM inference and drives GPU kernel optimization with Magpie.

    408 GitHub stars~2.3k tokensUpdated yesterday
    AI & LLM EngineeringAuto-check passed

  • Runpod

    ericrisco/rsc-harness

    A skill your agent uses when running GPU compute on RunPod and deciding between Pods (hourly, always-on) and Serverless (per-second, autoscaling) for training, fine-tuning or inference — serverless…

    180 GitHub stars~2.8k tokensUpdated yesterday
    AI & LLM EngineeringAuto-check passed

  • Modal

    K-Dense-AI/scientific-agent-skills

    Modal is a serverless cloud platform for running Python on demand, including on-demand GPUs.

    48k GitHub starsUsed in 1 repo~4.5k tokens
    Backend & APIsAuto-check: notes

More from Orchestra-Research/AI-Research-SKILLs

All 96 skills in this repo

  • AudioCraft Audio Generation

    Orchestra-Research/AI-Research-SKILLs

    Generates music from text descriptions with MusicGen and sound effects with AudioGen, using Meta's AudioCraft PyTorch library with melody and style conditioning.

    13k GitHub starsUsed in 8 repos~3.9k tokens
    Auto-check passed

  • LLM Benchmarking with lm-evaluation-harness

    Orchestra-Research/AI-Research-SKILLs

    Runs lm-evaluation-harness to benchmark language models on academic suites such as MMLU, GSM8K and HumanEval, compare models and track training checkpoints.

    13k GitHub starsUsed in 8 repos~3k tokens
    Auto-check passed

  • Segment Anything Model Guide

    Orchestra-Research/AI-Research-SKILLs

    Guide to using Meta's Segment Anything Model for zero-shot image segmentation with point, box or mask prompts, or automatic mask generation.

    13k GitHub starsUsed in 8 repos~3.3k tokens
    Auto-check passed

  • Chroma Vector Database

    Orchestra-Research/AI-Research-SKILLs

    Shows how to store documents and embeddings in Chroma, query them by similarity with metadata filters, and persist them to disk for RAG and semantic search projects.

    13k GitHub starsUsed in 7 repos~2.3k tokens
    Auto-check passed

  • CLIP Image-Text Matching

    Orchestra-Research/AI-Research-SKILLs

    Explains OpenAI's CLIP model for zero-shot image classification, image-text similarity, semantic image search and content moderation, with install steps and code patterns.

    13k GitHub starsUsed in 7 repos~1.7k tokens
    Auto-check passed

  • Whisper Speech Recognition

    Orchestra-Research/AI-Research-SKILLs

    Transcribes audio with OpenAI's Whisper: 99 languages, translation to English, language detection, six model sizes and word-level timestamps, from Python or the CLI.

    13k GitHub starsUsed in 7 repos~1.9k tokens
    Auto-check: notes

Works with

Categories

Questions about Axolotl Fine-Tuning Reference

What does Axolotl Fine-Tuning Reference do?

Quick-reference help for fine-tuning language models with Axolotl, covering YAML configs, FSDP, context parallelism, compressed saves and dataset formats. The skill is a reference pack drawn from Axolotl's official documentation. Its quick-reference section collects short patterns: running NCCL tests to check data-transfer speed between GPUs, enabling FSDP in the Axolotl YAML (with `fsdp_version` and `fsdp_config`), choosing a `context_parallel_size` that divides the number of GPUs, and setting `save_compressed: true` to save models in a smaller format that vLLM and llmcompressor can still load, reported as about 40% less disk use.

When should I use Axolotl Fine-Tuning Reference?

Axolotl Fine-Tuning Reference fits situations like: writing or debugging an Axolotl YAML config for a fine-tuning run; choosing a dataset format for Axolotl training data; setting up FSDP or context parallelism across several GPUs; looking up Axolotl API classes before extending the trainer.

How do I install Axolotl Fine-Tuning Reference in Claude Code?

Run `npx skills add Orchestra-Research/AI-Research-SKILLs --skill axolotl -a claude-code`. Or copy the skill folder (03-fine-tuning/axolotl in Orchestra-Research/AI-Research-SKILLs) into .claude/skills/axolotl in your project. Claude Code loads it when a task matches its description.

How do I install Axolotl Fine-Tuning Reference in Codex?

Run `npx skills add Orchestra-Research/AI-Research-SKILLs --skill axolotl -a codex`. Or copy the skill folder (03-fine-tuning/axolotl in Orchestra-Research/AI-Research-SKILLs) into .agents/skills/axolotl in your project. Codex loads it when a task matches its description.

Can I use Axolotl Fine-Tuning Reference in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add Orchestra-Research/AI-Research-SKILLs --skill axolotl -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/axolotl, .gemini/skills/axolotl, .github/skills/axolotl and .opencode/skills/axolotl in your project.

What does Axolotl Fine-Tuning Reference need to run?

SKILL.md names no scripts, command-line tools or credentials: Axolotl Fine-Tuning Reference is instructions for the agent only. Our summary lists: Axolotl installed in a Python environment; GPUs for training runs.

Does Axolotl Fine-Tuning Reference access the network?

SKILL.md names 1 domain. As links in the text: github.com. This is read from the text; nothing was executed.

Is Axolotl Fine-Tuning Reference safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.

What licence does Axolotl Fine-Tuning Reference use?

Axolotl Fine-Tuning Reference is published under the MIT licence (declared in SKILL.md). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Axolotl Fine-Tuning Reference use?

About 1.2k tokens (SKILL.md is roughly 4.8k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full. Its references folder adds about 77k tokens, read only when the agent opens those files.

What are the alternatives to Axolotl Fine-Tuning Reference?

Skills that share tags, products or a category with Axolotl Fine-Tuning Reference: Hugging Face Vision Trainer (huggingface/skills, 11k stars), Aqua Deployment (oracle/accelerated-data-science, 125 stars), LLM Serving Framework Benchmark (BBuf/AI-Infra-Auto-Driven-SKILLS, 938 stars) and Magpie Kernel Evaluator (amd/skills, 408 stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Axolotl Fine-Tuning Reference?

Orchestra-Research (a GitHub organization) maintains it in Orchestra-Research/AI-Research-SKILLs, which has 13,405 GitHub stars. The repository holds 96 skills in this directory. The repository was last updated on June 16, 2026.

Source: Orchestra-Research/AI-Research-SKILLs on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.