LLM App Builder
revfactory/harness-100
Full pipeline where an agent team collaborates to develop an LLM app.
Fine-tune models on Microsoft Foundry using SFT (supervised), DPO (preference), or RFT (reinforcement with graders).
Install Finetuning
$ npx skills add microsoft/GitHub-Copilot-for-Azure --skill finetuning -a claude-codeProject install by default; add -g for ~/.claude/skills/.
$ gh skill install microsoft/GitHub-Copilot-for-Azure finetuning --agent claude-codeProject scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).
$ git clone --depth 1 https://github.com/microsoft/GitHub-Copilot-for-Azure.git skills-src && mkdir -p .claude/skills && cp -r skills-src/plugins/azure-skills/skills/microsoft-foundry/finetuning .claude/skills/finetuning && rm -rf skills-srcUse ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.
Claude Code skills documentation · loads skills from .claude/skills/
Install the "finetuning" agent skill from https://github.com/microsoft/GitHub-Copilot-for-Azure/tree/main/plugins/azure-skills/skills/microsoft-foundry/finetuning into .claude/skills/finetuning/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "finetuning", then confirm the skill loads.Claude Code copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$skill-installer install https://github.com/microsoft/GitHub-Copilot-for-Azure/tree/main/plugins/azure-skills/skills/microsoft-foundry/finetuningType this inside Codex. $skill-installer <name> installs a curated skill from openai/skills. The installer writes to $CODEX_HOME/skills (default ~/.codex/skills). Restart Codex if the skill does not show up.
$ npx skills add microsoft/GitHub-Copilot-for-Azure --skill finetuning -a codexProject install goes to .agents/skills/; add -g for ~/.codex/skills/.
$ gh skill install microsoft/GitHub-Copilot-for-Azure finetuning --agent codexProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/microsoft/GitHub-Copilot-for-Azure.git skills-src && mkdir -p .agents/skills && cp -r skills-src/plugins/azure-skills/skills/microsoft-foundry/finetuning .agents/skills/finetuning && rm -rf skills-srcUse ~/.agents/skills/ instead of .agents/skills for a personal install.
Codex skills documentation · loads skills from .agents/skills/
Install the "finetuning" agent skill from https://github.com/microsoft/GitHub-Copilot-for-Azure/tree/main/plugins/azure-skills/skills/microsoft-foundry/finetuning into .agents/skills/finetuning/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "finetuning", then confirm the skill loads.Codex copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add microsoft/GitHub-Copilot-for-Azure --skill finetuning -a cursorProject install goes to .agents/skills/; add -g for ~/.cursor/skills/.
$ gh skill install microsoft/GitHub-Copilot-for-Azure finetuning --agent cursorProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/microsoft/GitHub-Copilot-for-Azure.git skills-src && mkdir -p .cursor/skills && cp -r skills-src/plugins/azure-skills/skills/microsoft-foundry/finetuning .cursor/skills/finetuning && rm -rf skills-srcUse ~/.cursor/skills/ instead of .cursor/skills for a personal install.
Cursor skills documentation · loads skills from .cursor/skills/, .agents/skills/, .claude/skills/, .codex/skills/
Install the "finetuning" agent skill from https://github.com/microsoft/GitHub-Copilot-for-Azure/tree/main/plugins/azure-skills/skills/microsoft-foundry/finetuning into .cursor/skills/finetuning/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "finetuning", then confirm the skill loads.Cursor copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gemini skills install https://github.com/microsoft/GitHub-Copilot-for-Azure.git --path plugins/azure-skills/skills/microsoft-foundry/finetuning--scope user (default) or --scope workspace; --path is the subfolder of the repo that holds the skill; --consent skips the security confirmation prompt.
$ npx skills add microsoft/GitHub-Copilot-for-Azure --skill finetuning -a gemini-cliProject install goes to .agents/skills/; add -g for ~/.gemini/skills/.
$ gh skill install microsoft/GitHub-Copilot-for-Azure finetuning --agent gemini-cliProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/microsoft/GitHub-Copilot-for-Azure.git skills-src && mkdir -p .gemini/skills && cp -r skills-src/plugins/azure-skills/skills/microsoft-foundry/finetuning .gemini/skills/finetuning && rm -rf skills-srcUse ~/.gemini/skills/ instead of .gemini/skills for a personal install, then run /skills reload.
Gemini CLI skills documentation · loads skills from .gemini/skills/, .agents/skills/
Install the "finetuning" agent skill from https://github.com/microsoft/GitHub-Copilot-for-Azure/tree/main/plugins/azure-skills/skills/microsoft-foundry/finetuning into .gemini/skills/finetuning/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "finetuning", then confirm the skill loads.Gemini CLI copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gh skill install microsoft/GitHub-Copilot-for-Azure finetuningInstalls for Copilot at project scope by default; add --scope user for a personal install. Preview a skill first with gh skill preview. Needs GitHub CLI 2.90.0 or later (public preview).
$ npx skills add microsoft/GitHub-Copilot-for-Azure --skill finetuning -a github-copilotProject install goes to .agents/skills/; add -g for ~/.copilot/skills/.
$ git clone --depth 1 https://github.com/microsoft/GitHub-Copilot-for-Azure.git skills-src && mkdir -p .github/skills && cp -r skills-src/plugins/azure-skills/skills/microsoft-foundry/finetuning .github/skills/finetuning && rm -rf skills-srcUse ~/.copilot/skills/ instead of .github/skills for a personal install. Commit .github/skills so cloud agent and code review can use it.
GitHub Copilot skills documentation · loads skills from .github/skills/, .claude/skills/, .agents/skills/
Install the "finetuning" agent skill from https://github.com/microsoft/GitHub-Copilot-for-Azure/tree/main/plugins/azure-skills/skills/microsoft-foundry/finetuning into .github/skills/finetuning/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "finetuning", then confirm the skill loads.GitHub Copilot copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add microsoft/GitHub-Copilot-for-Azure --skill finetuning -a opencodeOpenCode documents no install command of its own. Project install goes to .agents/skills/; add -g for ~/.config/opencode/skills/.
$ gh skill install microsoft/GitHub-Copilot-for-Azure finetuning --agent opencodeProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/microsoft/GitHub-Copilot-for-Azure.git skills-src && mkdir -p .opencode/skills && cp -r skills-src/plugins/azure-skills/skills/microsoft-foundry/finetuning .opencode/skills/finetuning && rm -rf skills-srcUse ~/.config/opencode/skills/ instead of .opencode/skills for a personal install.
OpenCode skills documentation · loads skills from .opencode/skills/, .claude/skills/, .agents/skills/
Install the "finetuning" agent skill from https://github.com/microsoft/GitHub-Copilot-for-Azure/tree/main/plugins/azure-skills/skills/microsoft-foundry/finetuning into .opencode/skills/finetuning/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "finetuning", then confirm the skill loads.OpenCode copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
Facts
- Skill name
finetuning- GitHub stars
- 255
- Used in
- 1 other repo
- Token cost
- ~1.4k tokens
- SKILL.md length
- 411 words
- Files
- 36 (incl. scripts, references)
- Skills in repo
- 56
- Repo updated
- First seen
- Licence
- MIT
At a glance
Fine-tune models on Microsoft Foundry using SFT (supervised), DPO (preference), or RFT (reinforcement with graders).
- Works in 5 steps: Always baseline first — evaluate the… → Validate data before submitting — run… → Calibrate RFT graders — target 25-50%… → …
- Fine-tuned model
- SKILL.md covers When to Use, Workflows, References and Scripts, plus 3 more sections
- Runs Python scripts from its folder; calls python
What it does
Finetuning is an agent skill from microsoft/GitHub-Copilot-for-Azure, published by the product's own GitHub organization. Fine-tune models on Microsoft Foundry using SFT (supervised), DPO (preference), or RFT (reinforcement with graders). Covers dataset preparation, training job submission, deployment, and evaluation. USE FOR: fine-tune, SFT, DPO, RFT, training data, grader, distillation, fine-tuned model, training job, large file upload, calibrate grader, deploy fine-tuned model, evaluate fine-tuned model. DO NOT USE FOR: general model deployment without fine-tuning (use deploy-model), agent creation (use agents), prompt…
Its SKILL.md is about 1.4k tokens, which your agent loads only when the skill is triggered. The skill folder holds 37 other files, including scripts and reference files (for example `references/agentic-rft.md`, `references/dataset-formats.md` and `references/deployment.md`).
It sits in AI & LLM Engineering, covering Fine-tuning, Prompt engineering and Deployment. The repository describes itself as: GitHub Copilot for Azure. The licence is MIT.
When your agent uses it
- Fine-tuned model
- Large file upload
- Calibrate grader
- Deploy fine-tuned model
Example prompts
- “/finetuning”
Requirements
- Python 3
Workflow steps
5 steps, taken from the first numbered list in SKILL.md.
- Always baseline first — evaluate the base model before fine-tuning
- Validate data before submitting — run scripts/validate/validate_sft.py
- Calibrate RFT graders — target 25-50% failure rate on the base model
- Evaluate checkpoints — don't blindly deploy the final one
- Measure token cost alongside accuracy when comparing models
What it can do on your machine
Read from SKILL.md and the folder at commit d8f4f4e. It shows what the files ask for, not the result of running them.
Tool permissions
Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.
From allowed-tools in the SKILL.md frontmatter.
Runs code
Ships 6 files in scripts/ (Python, from the files we listed), which the agent can run.
Shell commands in SKILL.md call:
python
From the folder's file list and the shell code blocks in SKILL.md.
Network
No URLs in SKILL.md.
From URLs in SKILL.md, links to its own repository left out.
Credentials
Names no API keys, tokens, secrets or passwords.
From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.
Context cost
Finetuning loads about 1.4k tokens when it runs, and up to ~12k if it reads all its reference files. Until then it costs about 143 tokens; SKILL.md has 411 words of instructions outside code blocks.
Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.
Safety
Auto-check passed
The automated check found no risky patterns in SKILL.md.
Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); the scripts in this folder are not scanned.
SKILL.md
The full file from microsoft/GitHub-Copilot-for-Azure at commit d8f4f4e, republished under its MIT licence (© microsoft). 411 words, ~1,376 tokens.
Download SKILL.mdSave it as
.claude/skills/finetuning/SKILL.md (or your agent's skills folder). This skill also uses 35 other files; get the full folder from GitHub.- name
- finetuning
- description
- Fine-tune models on Microsoft Foundry using SFT (supervised), DPO (preference), or RFT (reinforcement with graders). Covers dataset preparation, training job submission, deployment, and evaluation. USE FOR: fine-tune, SFT, DPO, RFT, training data, grader, distillation, fine-tuned model, training job, large file upload, calibrate grader, deploy fine-tuned model, evaluate fine-tuned model. DO NOT USE FOR: general model deployment without fine-tuning (use deploy-model), agent creation (use agents), prompt optimization without training (use prompt-optimizer).
- license
- MIT
- metadata.author
- Microsoft
- metadata.version
- 0.0.0-placeholder
Fine-Tuning on Microsoft Foundry
Fine-tune models using SFT (supervised), DPO (preference), or RFT (reinforcement with graders). Covers dataset prep, training, deployment, and evaluation.
When to Use
Use this sub-skill when the user asks about:
- Fine-tuning a model (SFT, DPO, or RFT)
- Preparing, validating, or formatting training data
- Submitting, monitoring, or diagnosing training jobs
- Calibrating graders or pass thresholds for RFT
- Deploying or evaluating a fine-tuned model
- Choosing between training types (SFT vs DPO vs RFT)
- Distillation, synthetic data generation, or dataset quality scoring
- Large file uploads for training data
- Cleaning up fine-tuning resources (files, deployments)
Do NOT use for: General model deployment without fine-tuning (use deploy-model), agent creation (use agents), prompt optimization without training (use prompt-optimizer).
Workflows
| Stage | Guide |
|---|---|
| Quick start | workflows/quickstart.md |
| Full pipeline | workflows/full-pipeline.md |
| Create data | workflows/dataset-creation.md |
| Iterate | workflows/iterative-training.md |
| Diagnose | workflows/diagnose-poor-results.md |
References
| Topic | File |
|---|---|
| SFT vs DPO vs RFT | references/training-types.md |
| Hyperparameters | references/hyperparameters.md |
| Data formats | references/dataset-formats.md |
| Grader design (RFT) | references/grader-design.md |
| Reward hacking | references/reward-hacking.md |
| Agentic RFT (tools) | references/agentic-rft.md |
| Deployment | references/deployment.md |
| Training curves | references/training-curves.md |
| Evaluation | references/evaluation.md |
| Vision fine-tuning | references/vision-fine-tuning.md |
| Large file uploads | references/large-file-uploads.md |
| Platform gotchas | references/platform-gotchas.md |
Scripts
| Script | Purpose |
|---|---|
scripts/submit_training.py | Submit SFT/DPO/RFT jobs |
scripts/monitor_training.py | Poll job until completion |
scripts/calibrate_grader.py | Find optimal RFT pass_threshold |
scripts/check_training.py | Analyze curves, list checkpoints |
scripts/deploy_model.py | Deploy via ARM REST API |
scripts/evaluate_model.py | LLM judge evaluation |
scripts/convert_dataset.py | Convert between SFT/DPO/RFT formats |
scripts/generate_distillation_data.py | Generate synthetic training data |
scripts/score_dataset.py | Quality scoring on training data |
scripts/cleanup.py | Delete old files and deployments |
scripts/validate/ | Data validators (SFT, DPO, RFT) + stats |
Show full SKILL.md (176 more words)Show less
Rules
- Always baseline first — evaluate the base model before fine-tuning
- Validate data before submitting — run
scripts/validate/validate_sft.py - Calibrate RFT graders — target 25-50% failure rate on the base model
- Evaluate checkpoints — don't blindly deploy the final one
- Measure token cost alongside accuracy when comparing models
Quick Reference
| Task | Command |
|---|---|
| Validate SFT data | python scripts/validate/validate_sft.py data.jsonl |
| Submit SFT job | python scripts/submit_training.py --model gpt-4.1-mini --training-file train.jsonl --validation-file val.jsonl --type sft |
| Monitor job | python scripts/monitor_training.py --job-id ftjob-xxx |
| Analyze curves | python scripts/check_training.py --job-id ftjob-xxx |
| Deploy model | python scripts/deploy_model.py --model-id ft:gpt-4.1-mini:... --name my-eval |
| Evaluate model | python scripts/evaluate_model.py --deployment-name my-eval --test-file test.jsonl |
Error Handling
| Error | Cause | Fix |
|---|---|---|
| "API version not supported" | Older openai SDK on /v1/ endpoint | Upgrade to openai>=1.0 |
| "does not support fine-tuning with Standard TrainingType" | OSS model needs globalStandard | Use --use-rest flag or script auto-falls back |
| Job stuck in post-training eval | Under-provisioned tool endpoint (RFT) | Scale to S2+, enable Always On |
| "DeploymentNotReady" after ARM succeeds | ARM/data-plane race condition | Delete and recreate deployment, wait 5 min |
| Content safety block at deployment | PII-dense training data | Remove problematic document types |
© microsoft, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file
Files
SKILL.md and 35 other files (scripts, references) in plugins/azure-skills/skills/microsoft-foundry/finetuning of microsoft/GitHub-Copilot-for-Azure.
- SKILL.md
- references/agentic-rft.md
- references/dataset-formats.md
- references/deployment.md
- references/evaluation.md
- references/grader-design.md
- references/hyperparameters.md
- references/large-file-uploads.md
- references/platform-gotchas.md
- references/reward-hacking.md
- references/training-curves.md
- references/training-types.md
- references/vision-fine-tuning.md
- scripts/calibrate_grader.py
- scripts/check_training.py
- scripts/cleanup.py
- scripts/common.py
- scripts/convert_dataset.py
- scripts/deploy_model.py
- … and 17 more
Open the folder on GitHubat commit d8f4f4e
Used in 3 other repositories
We found 4 copies of this SKILL.md (exact, near-identical or edited) in other folders, from 1 other GitHub owner. This page covers the copy in microsoft/GitHub-Copilot-for-Azure, which our catalogue first saw on October 7, 2026.
Compare with similar skills
Finetuning next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.
| Skill | Stars | Used in | Tokens | Auto-check | Licence | Repo updated |
|---|---|---|---|---|---|---|
| Finetuning this skillmicrosoft/GitHub-Copilot-for-Azure | 255 | 1 repos | ~1.4k | Automated safety check: Pass | MIT | |
| LLM App Builderrevfactory/harness-100 | 1.3k | — | ~1.9k | Automated safety check: Pass | Apache-2.0 | |
| AWS AI MLaws/agent-toolkit-for-aws | 2.8k | — | ~1.7k | Automated safety check: Pass | Apache-2.0 | |
| Agent OrchestrationLeoYeAI/openclaw-master-skills | 2.2k | — | ~4.4k | Automated safety check: Pass | MIT | |
| Aqua CLIoracle/accelerated-data-science | 125 | — | ~2.1k | Automated safety check: Pass | UPL-1.0 | |
| Agent Platform Eval Flywheelgoogle/skills | 21k | — | ~6.7k | Automated safety check: Pass | Apache-2.0 |
Similar skills
- Official
AWS AI ML
aws/agent-toolkit-for-aws
Selects, deploys, and customizes AI models on Amazon SageMaker.
Agent Orchestration
LeoYeAI/openclaw-master-skills
Multi-agent orchestration patterns for production deployments.
- Official
Aqua CLI
oracle/accelerated-data-science
Complete CLI reference for the ADS AQUA command-line interface (ads aqua).
- Official
Agent Platform Eval Flywheel
google/skills
Measures and improves the quality of AI models and agents on Google Cloud using the Eval Quality Flywheel methodology.
Quantized Export
wshobson/agents
Export a promoted fine-tuned model in the right deployment format — merged safetensors, LoRA-only, GGUF with imatrix, or FP8.
More from microsoft/GitHub-Copilot-for-Azure
All 56 skills in this repo- Official
Capacity
microsoft/GitHub-Copilot-for-Azure
Discovers available Azure OpenAI model capacity across regions and projects.
- Official
Deploy Model
microsoft/GitHub-Copilot-for-Azure
Unified Azure OpenAI model deployment skill with intelligent intent-based routing.
- Official
Entra Agent Id
microsoft/GitHub-Copilot-for-Azure
Provision Microsoft Entra Agent Identity Blueprints, BlueprintPrincipals, and per-instance Agent Identities via Microsoft Graph, and configure OAuth 2.0 token exchange (fmipath, OBO, cross-tenant)…
- Official
Microsoft Foundry
microsoft/GitHub-Copilot-for-Azure
Build, deploy, evaluate, optimize, fine-tune, and manage Microsoft Foundry agents, models, and resources end to end.
- Official
Azure Storage
microsoft/GitHub-Copilot-for-Azure
Azure Storage Services including Blob Storage, File Shares, Queue Storage, Table Storage, and Data Lake.
- Official
Azure Diagnostics
microsoft/GitHub-Copilot-for-Azure
Debug Azure production issues on Azure using AppLens, Azure Monitor, resource health, and safe triage.
Related
Categories
Questions about Finetuning
What does Finetuning do?
Fine-tune models on Microsoft Foundry using SFT (supervised), DPO (preference), or RFT (reinforcement with graders). Finetuning is an agent skill from microsoft/GitHub-Copilot-for-Azure, published by the product's own GitHub organization. Fine-tune models on Microsoft Foundry using SFT (supervised), DPO (preference), or RFT (reinforcement with graders).
When should I use Finetuning?
Finetuning fits situations like: fine-tuned model; large file upload; calibrate grader; deploy fine-tuned model.
How do I install Finetuning in Claude Code?
Run `npx skills add microsoft/GitHub-Copilot-for-Azure --skill finetuning -a claude-code`. Or copy the skill folder (plugins/azure-skills/skills/microsoft-foundry/finetuning in microsoft/GitHub-Copilot-for-Azure) into .claude/skills/finetuning in your project. Claude Code loads it when a task matches its description.
How do I install Finetuning in Codex?
Run `npx skills add microsoft/GitHub-Copilot-for-Azure --skill finetuning -a codex`. Or copy the skill folder (plugins/azure-skills/skills/microsoft-foundry/finetuning in microsoft/GitHub-Copilot-for-Azure) into .agents/skills/finetuning in your project. Codex loads it when a task matches its description.
Can I use Finetuning in Cursor, Gemini CLI or GitHub Copilot?
Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add microsoft/GitHub-Copilot-for-Azure --skill finetuning -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/finetuning, .gemini/skills/finetuning, .github/skills/finetuning and .opencode/skills/finetuning in your project.
What does Finetuning need to run?
Going by SKILL.md and its folder, Finetuning needs Python for the scripts in its folder and the command-line tools its instructions call (python). Our summary lists: Python 3.
Does Finetuning access the network?
SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.
Is Finetuning safe to install?
Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. The check reads SKILL.md only: the scripts in the folder are not scanned, so read them before running anything.
What licence does Finetuning use?
Finetuning is published under the MIT licence (declared in SKILL.md). It allows redistribution, so the full SKILL.md is shown on this page.
How many tokens does Finetuning use?
About 1.4k tokens (SKILL.md is roughly 5.5k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full. Its references folder adds about 10k tokens, read only when the agent opens those files.
What are the alternatives to Finetuning?
Skills that share tags, products or a category with Finetuning: LLM App Builder (revfactory/harness-100, 1.3k stars), AWS AI ML (aws/agent-toolkit-for-aws, 2.8k stars), Agent Orchestration (LeoYeAI/openclaw-master-skills, 2.2k stars) and Aqua CLI (oracle/accelerated-data-science, 125 stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.
Who maintains Finetuning?
microsoft (a GitHub organization, an official publisher) maintains it in microsoft/GitHub-Copilot-for-Azure, which has 255 GitHub stars. The repository holds 56 skills in this directory. The repository was last updated on October 7, 2026.
Source: microsoft/GitHub-Copilot-for-Azure on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.