Reviewing Skills

Root-causing trigger failures

When a skill or instruction existed but you did not follow it, treat that as a defect in the instructions — not just a one-off mistake. The instruction failed to trigger at the right moment, and the fix is to understand why and close the gap.

Ask these questions:

1. Was the instruction framed for the wrong trigger? An instruction that says "when you edit a documentation file" may not fire when the task is "fix the CI build." If the instruction applies in both situations, rewrite it to cover both entry points.
2. Was the instruction buried inside a larger workflow? A critical step hidden as step 5 of a 5-step process gets skipped when you jump to the end. Promote it to a standalone gate with its own heading.
3. Did the instruction assume proactive compliance? Instructions that only describe the happy path ("do X when you change Y") need a reactive counterpart ("if you forgot to do X, here is how to recover — and it still requires doing X").
4. Did multi-turn conversation obscure the trigger? When changes accumulate across many conversation turns before a commit, per-edit triggers get lost. Anchor the instruction to the commit point instead, since that is where all changes converge regardless of how they were made.

After root-causing, update the instruction to close the gap — then verify the updated instruction would have caught the original failure.

Running a full review

For a periodic audit of all skills (e.g., after a major release):

1. List all skills and check each against the current source
2. Use parallel explore agents to verify 4-5 skills each — look for fabricated APIs, wrong parameters, stale paths, missing examples
3. Cross-check findings against source code before fixing — review agents produce false positives (they may flag correct code as wrong)
4. Track findings in a structured format (SQL table or similar) with severity and status
5. Fix in priority order: critical inaccuracies → missing examples → duplication → cosmetic

Common false positives

Review agents frequently flag things that are actually correct:

• Generated type APIs that don't appear in hand-written source (they're emitted by the code generator)
• Methods found only via generic type inference (the agent's grep misses them)
• Shared source files in src-v4/ that are referenced by V5 projects via project references

Always verify against the actual source before changing a skill.

Design principles for skill content

These principles keep skills effective as AI context:

• Code-first — every skill should have at least one copy-paste-ready code example with real syntax
• Concrete over abstract — real file paths, real type names, real method signatures
• Self-contained sections — each section should make sense without reading the whole skill
• Tables for reference data — configuration options, threshold values, priority levels
• Pitfalls earn their place — only document pitfalls that have actually caused problems

Cross-References

• For verifying code samples in documentation, see the "Documentation Code Sample Verification" section in copilot-instructions.md
• For the code sample catalog tools, see docs/CodeSampleCatalog.md