Docs Corpus Audit

What it does

When your agent uses it

• Run a holistic regrounding pass on the entire microsoft/apm documentation corpus against current source code
• Emit surgical fixes for stale claims
• Include audit the docs
• Reground the corpus

Example prompts

• “audit the docs”
• “reground the corpus”
• “check every page against code”
• “/docs-corpus-audit”

Requirements

• Python 3
• A Bash shell

What it can do on your machine

Read from SKILL.md and the folder at commit 280b8a7. It shows what the files ask for, not the result of running them.

• Tool permissions

  Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

  From allowed-tools in the SKILL.md frontmatter.

• Runs code

  Ships 1 file in scripts/ (Shell), which the agent can run.

  Shell commands in SKILL.md call:

  • git
  • uv
  • python
  • gh

  From the folder's file list and the shell code blocks in SKILL.md.

• Network

  No URLs in SKILL.md. Its commands use git, uv and gh, which can reach the network depending on how they are called.

  From URLs in SKILL.md, links to its own repository left out.

• Credentials

  Names no API keys, tokens, secrets or passwords.

  From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Docs Corpus Audit loads about 2.6k tokens when it runs. Until then it costs about 233 tokens; SKILL.md has 857 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it

~233

When it runs · the whole SKILL.md, loaded when a task matches

~2.6k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); the scripts in this folder are not scanned.

SKILL.md

The full file from microsoft/apm at commit 280b8a7, republished under its MIT licence (© microsoft). 857 words, ~2,551 tokens.

name

docs-corpus-audit

description

Use this skill to run a holistic regrounding pass on the entire microsoft/apm documentation corpus against current source code, page-by-page, and emit surgical fixes for stale claims. Activate when the maintainer wants a WHOLE-CORPUS audit (not per-PR review) -- typical triggers include "audit the docs", "reground the corpus", "check every page against code", "pre-release docs sweep", "the docs have drifted everywhere", or "we just reshaped the TOC, find dead links". Wave-batched and S7-verified; scales to the full ~112-page corpus in ~10 minutes wall-time. This is a SIBLING to docs-sync, not a replacement: docs-sync is per-PR (triggered by a diff); this skill is per-corpus (triggered by a maintainer ask). They share agent personas, schemas, and the docs index, but their triggers MUST NOT collide. Does NOT auto-merge, does NOT push without maintainer review, and does NOT replace per-PR drift detection.

1. PROBE (A9 SUPERVISED EXECUTION)
   - Check working tree: docs/src/content/docs/ exists?
   - Check working tree: packages/apm-guide/.apm/skills/apm-usage/
     exists? (Rule-4 backfill target. If missing, the audit cannot
     close Rule 4; ask maintainer before continuing.)
   - Check `.apm/docs-index.yml` reachable.
   - Verify on a working branch (not main).

2. RISK-TRIAGE (orchestrator, ~1 LLM call)
   - Read .apm/docs-index.yml only (NOT the corpus body).
   - Bucket pages by drift risk: HIGH (CLI ref, schemas, consumer
     flows), MEDIUM (producer, enterprise policy), LOW (concepts,
     contributing, troubleshooting, integrations).
   - Decide wave order: HIGH first, MEDIUM next, LOW last.

3. WAVE-PLANNER (orchestrator, deterministic)
   - Partition pages into 6-8 disjoint scopes per wave.
   - Each agent gets ~9 pages, mixed surface types.

4. WAVE EXECUTION (parallel, one subagent per scope)
   - Orchestrator dispatches one task per scope using the prompt
     template in assets/subagent-prompt-template.md.
   - Subagents read pages, extract claims, S7-verify, apply
     surgical edits, return JSON per the docs-sync panelist
     schema (mirrored at assets/panelist-return-schema.json).
   - Validate every return against the schema; reject malformed
     JSON.

5. CROSS-CORPUS POST-PASS (orchestrator, deterministic)
   - Run scripts/scan-cross-corpus-drift.sh to grep for patterns
     a per-scope agent cannot see (IA-reshuffle dead links, stale
     deprecation version targets, phantom flag references).
   - Patch residue inline.

6. ALIGNMENT LOOP (orchestrator, deterministic)
   - Re-run scripts/scan-cross-corpus-drift.sh.
   - Re-grep for claims the agents marked DRIFTED-FIXED.
   - If residue: targeted re-dispatch to the owning agent
     (bounded: max 2 redrafts per wave).

7. COMMIT + PUSH (orchestrator, single writer)
   - One commit per wave; structured message naming closed items.
   - Push to working branch.

8. PR + SUMMARY COMMENT (orchestrator)
   - If no PR exists: open one with the [pr-description-skill]
     (../pr-description-skill/SKILL.md).
   - Post per-wave summary comment: pages audited, drift caught,
     fixes applied, items deferred, alignment-loop residue.

© microsoft, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

SKILL.md and 6 other files (scripts, assets) in .apm/skills/docs-corpus-audit of microsoft/apm.

• SKILL.md
• assets/panelist-return-schema.json
• assets/subagent-prompt-template.md
• evals/README.md
• evals/content-evals.json
• evals/trigger-evals.json
• scripts/scan-cross-corpus-drift.sh

Open the folder on GitHubat commit 280b8a7