Skill Auditor

Skill Auditor

Audits every SKILL.md in skills/ for frontmatter correctness, structural completeness, API/version drift, and cross-file reference integrity. Produces a prioritized markdown report. Read-only — never applies fixes.

When This Skill Activates

Use this skill when the user:

• Says "audit skills", "audit my skills", "run the skill auditor"
• Asks to "check the skill repo for drift", "review skills for consistency"
• Says "find broken skills", "validate skill frontmatter"
• Is planning a bulk cleanup of the skill repo
• References a specific category to audit (e.g., "audit the generators/ skills")

Does NOT activate for: creating new skills (use skill-creator), applying fixes (follow-up flow after this auditor reports), or auditing Swift code inside skills (use ios/coding-best-practices or macos/coding-best-practices).

Scope

Resolve the invocation argument in this order:

1. No argument → audit all of skills/**/SKILL.md
2. Category path (e.g., generators/, ios/) → audit skills/<arg>/**/SKILL.md
3. Single file path (e.g., skills/liquid-glass/SKILL.md) → audit one file
4. Fuzzy match → if arg doesn't resolve, try skills/<arg>/SKILL.md; fall back to asking the user

Process

1. Enumerate

Use Glob with pattern skills/**/SKILL.md from the repo root. Filter by scope if an argument was passed. Record the canonical file list — every subsequent step operates on this list.

2. Parse Frontmatter (Cached)

Read the first 20 lines of each SKILL.md. Parse:

• Whether --- frontmatter block exists
• name:, description:, allowed-tools: field values
• Freshness keys: last_verified:, review_by: (dates, required — enforced by scripts/check-frontmatter.sh), optional os_version:. Overdue review_by dates are the stale-skills workflow's job, not a finding here.

Cache this result. Checks C-01, H-01, H-03, L-02 all read from this cache — do not re-read.

3. Run Checks

Execute bulk Grep passes in parallel (single tool-call batch) wherever possible. Per-file operations come after. The 11 checks are in the table below; the order is as-listed.

4. Classify Aggregator vs Leaf

For each file, mark it as aggregator or leaf using the rules in the "Aggregator Detection" section below. Some checks relax for aggregators.

5. Rank and Emit

Group findings by severity (🔴 → 🟢), sort within each group by file path, print the report inline using the template in "Output Format".

Checks

🔴 Critical

C-01 · Missing frontmatter. The file has no leading --- YAML block.

• Detection: multiline Grep for \A---\n[\s\S]*?\n--- across all SKILL.md. Files with no match → C-01.
• Fix: Add YAML frontmatter with name, description, allowed-tools.

🟠 High

H-01 · Missing allowed-tools field. Frontmatter exists but allowed-tools: key is absent.

• Detection: from cached frontmatter parse.
• Fix: Add allowed-tools: [Read, Glob, Grep] (adjust based on what the skill actually does).

H-02 · Broken supporting-file reference. The SKILL.md references a *.md file that does not exist on disk in the same directory.

• Detection: Grep each SKILL.md for [a-z0-9][a-z0-9-]*\.md matches; resolve each relative to the SKILL.md's directory; ls to confirm. Missing files → H-02. Ignore matches inside fenced code blocks. Also resolve rules/swiftlint.yml references the same way (fragment fixtures themselves are CI's job — scripts/check-lint-fragments.sh).
• Fix: Create the file or remove the reference.

H-03 · H1 title does not match name:. The first # heading after the frontmatter, slugified (lowercase, spaces → -), differs from the name: field.

• Detection: from cached frontmatter parse + per-file line-after-frontmatter.
• Fix: Rename the H1 or the name: to match.

🟡 Medium

M-01 · Missing "When This Skill Activates" section. No ## When This Skill Activates heading anywhere in the file.

• Detection: Grep -L for ^## When This Skill Activates across all SKILL.md.
• Fix: Add section with 3–5 user trigger phrases. See shared/skill-creator/SKILL.md for the canonical format.

M-02 · Outdated version reference (drift). Mentions iOS 17–25, macOS 13–25, or Swift 5.x with drift context (treated as current/latest/target).

• Detection (two-stage):
  1. Stage 1 — Grep for \biOS (1[7-9]|2[0-5])\b|\bSwift 5\.\d+\b|\bmacOS (1[3-9]|2[0-5])\b, capturing line numbers.
  2. Stage 2 — for each hit, examine ±2 surrounding lines. Classify:
     • Flag M-02 if surrounding lines contain: latest, newest, current, target, deployment target, requires, minimum, as of, new in, now supports, today
     • Suppress if surrounding lines contain: legacy, pre-, prior to, before, deprecated, old, migrate from, backport, fallback, if available, #available, or the version mention has a trailing + (e.g., iOS 17+)
     • Neither → L-03 (ambiguous)
• Known-current constants (dated 2026-07-11, update per WWDC): iOS 27, macOS 27 (announced WWDC26; iOS 26 / macOS 26 are within the one-generation grace), Swift 6.x. Normative source: scripts/versions.env. Mentions of these with drift context are always clean.
• Fix: Update the reference to the current generation (see scripts/versions.env), or annotate as legacy context with one of the suppression keywords.

M-03 · Pre-@Observable pattern without deprecation callout. Uses @StateObject or ObservableObject without acknowledging that @Observable is the current pattern.

• Detection: Grep for @StateObject|ObservableObject with line numbers; for each hit, secondary Grep of the same file for @Observable|deprecated|legacy|pre-@Observable|migration|old pattern within ±10 lines. No secondary match → M-03.
• Fix: Either replace with @Observable + @State, or add a migration note explaining why the older pattern is shown.

M-04 · Oversized single-file skill. SKILL.md exceeds 400 lines and its directory contains no sibling .md files.

• Detection: wc -l via Bash on each SKILL.md; if >400, check sibling file list via ls for any other .md. None → M-04.
• Fix: Modularize — extract sections into patterns.md, templates.md, checklist.md, or examples.md per skill-creator conventions.

🟢 Low

L-01 · No ✅/❌ examples in prose. The file has no ✅ or ❌ markers anywhere.

• Detection: Grep -L for ✅|❌.
• Fix: Add at least one good/bad example pair.

L-02 · Description length out of range. description: is <20 or >300 characters.

• Detection: from cached frontmatter parse.
• Fix: Expand or shorten the description. Include a "use when…" clause to anchor activation.

L-03 · Ambiguous version mention. A version keyword matched stage 1 of the drift check but surrounding lines contained neither drift nor legacy context. User reviews manually.

• Detection: fallthrough from M-02 stage-2 classification.
• Fix: Add a drift or legacy keyword to disambiguate, or leave as-is if the context is clearly a one-off mention.

Aggregator vs Leaf Detection

A SKILL.md is an aggregator if any of the following hold:

• It sits at depth 2 under the repo root — i.e., skills/<category>/SKILL.md
• It contains the heading ## Available Modules or ## Available Skills
• It links to ./<subdir>/SKILL.md or contains two or more references of the form skills/<category>/<subskill>/

Otherwise it is a leaf.

Relaxations for aggregators

Tag every finding in the report with (aggregator) or (leaf) so severity can be read at a glance.

Output Format

Print the report inline to the conversation using this template. Use exact headings — downstream tooling may grep them.

# Skill Audit Report — <YYYY-MM-DD> — <N> files scanned

## Summary
- 🔴 Critical: <count>
- 🟠 High: <count>
- 🟡 Medium: <count>
- 🟢 Low: <count>
- ✅ Files clean: <clean-count> / <N>

Scope: <all | category | single file>

## 🔴 Critical Findings
### C-01 · Missing frontmatter
- `skills/<path>/SKILL.md` (leaf) — **Fix:** Add YAML frontmatter with `name`, `description`, `allowed-tools`.

## 🟠 High Findings
### H-01 · Missing `allowed-tools` field
- `skills/liquid-glass/SKILL.md:1-4` (leaf) — **Fix:** Add `allowed-tools: [Read, Glob, Grep]`.
- `skills/macos/macos-tahoe-apis/SKILL.md:1-4` (leaf) — **Fix:** Same.

### H-02 · Broken supporting-file reference
- `skills/<path>/SKILL.md:<line>` — references `patterns.md`, not found. **Fix:** Create the file or remove the reference.

### H-03 · H1 title does not match `name:`
- `skills/<path>/SKILL.md` — `name: foo-bar`, H1 is `# Foo Bars`. **Fix:** Rename one to match.

## 🟡 Medium Findings
### M-01 · Missing "When This Skill Activates" section (<count>)
- `skills/design/liquid-glass/SKILL.md` (leaf)
- `skills/macos/coding-best-practices/SKILL.md` (leaf)
- [collapsed list of remaining offenders]
- **Fix:** Add section with 3–5 user trigger phrases.

### M-02 · Outdated version reference (<count>)
- `skills/<path>/SKILL.md:87` (leaf) — "latest iOS 17" in drift context. **Fix:** Update to iOS 26.

### M-03 · Pre-`@Observable` pattern without callout (<count>)
- `skills/<path>/SKILL.md:142` (leaf) — `@StateObject` without migration note. **Fix:** Replace with `@Observable` or add callout.

### M-04 · Oversized single-file skill (<count>)
- `skills/<path>/SKILL.md` — <NNN> lines, no sibling .md files. **Fix:** Modularize into `patterns.md` / `templates.md`.

## 🟢 Low Findings
### L-01 · No ✅/❌ examples
- `skills/<path>/SKILL.md` (leaf) — **Fix:** Add at least one good/bad example pair.

### L-02 · Description length out of range
- `skills/<path>/SKILL.md` — description is <N> chars. **Fix:** Expand/shorten to 20–300 chars.

### L-03 · Ambiguous version mention
- `skills/<path>/SKILL.md:<line>` — "iOS 18" with no drift/legacy context. **Fix:** Review manually.

## ✅ Clean Files
<collapsed list of files that passed all checks>

## Next Steps
- Review 🔴/🟠 findings first — they block skills from working as intended.
- Batch-fix 🟡 M-01 and M-02 mechanically — suggested find/replace scripts shown per-file.
- 🟢 findings are polish; address opportunistically.