Harness Audit

Harness Audit

Trigger on phrases like: "audit my harness", "evaluate my agent setup", "score my CLAUDE.md", "is my project ready for long-run", "5-subsystem assessment", "what's missing from my project setup", "/harness-audit". Run proactively when joining an unfamiliar codebase that has agent artifacts (CLAUDE.md, .claude/, AGENTS.md) but obvious gaps. Skip for single-file scripts and pure exploration.

Score a project's agent harness across five subsystems and tell the user which evidenced bottleneck to address first. Distinguish an artifact's presence from demonstrated behavior; never present metadata alone as runtime proof.

Source: Five-subsystem framework adapted from Learn Harness Engineering (walkinglabs, MIT). Adapted to our concrete stack: CLAUDE.md, .claude/rules/, PROBLEMS.md, feature_list.json, init.sh, hooks, handoffs, chronicles.

What This Skill Does

Given a project directory, produces a scorecard like this. This example assumes project-xyz delivers features across sessions and uses pull requests:

=== Harness Audit: project-xyz ===

Instructions  4/5  ✓ Agent entrypoint and modular rules are documented and used
                   ~ PR review guidance is not found in the inspected evidence
State         2/5  ✓ 3 handoffs preserve some continuation state
                   ✗ No current record locates active scope and deferred work
                   ~ PROBLEMS.md / feature_list.json are suitable conventions, not prerequisites
Verification  3/5  ✓ Documented test command; pytest is configured
                   ~ No current execution receipt supplied
                   ✗ No documented staged validation/proof route
Scope         3/5  ✓ in-scope principle in CLAUDE.md
                   ~ shared-resource policy is unknown; no serialized lane is declared
                   ✗ Definition of Done not explicit
Lifecycle     2/5  ✗ Needed session-boundary entry/stop behavior is not documented or demonstrated
                   ~ Manual cleanup convention exists but not enforced

Bottleneck: State (2/5) — no current locator for feature scope and deferred work

Priority improvement (only when the user asks for recommendations):
- Record an execution receipt for the existing test command   ↗ Verification evidence

For an audit-only request, this skill produces the scorecard without making changes or running probes merely to convert unknown into a pass. If the user also requested correction or implementation, the scorecard is an intermediate result: return the confirmed findings to the owning task and execute its necessary, authorized reversible fixes, verification and delivery. Do not stop at a report or assign agent-owned fixes back to the user. Preserve the original acceptance criteria and real external/irreversible boundaries.

The Five Subsystems (Our Adaptation)

See references/checklist-per-subsystem.md for per-subsystem concrete checks. See references/scoring-rubric.md for how to interpret 1-5 scores.

How to Run an Audit

Phase 1 — Gather

Read these files in order (skip silently if missing):

1. CLAUDE.md in project root
2. AGENTS.md in project root (some projects use this name)
3. .claude/rules/*.md (project-level rules)
4. .claude/settings.json and .claude/settings.local.json (hooks config)
5. PROBLEMS.md in root
6. feature_list.json in root
7. init.sh in root (and Makefile / package.json scripts as fallback)
8. .claude/handoffs/ (count files, check INDEX.md existence)
9. .claude/chronicles/ (count files)
10. Sample test config: pytest.ini / package.json test script / Cargo.toml

Use Glob + Read for the harness, then inspect the smallest relevant evidence path: a current test/CI receipt, hook execution trace, or sampled state artifact. This is not a broad code review; absence of behavioral evidence is ~ unknown, not ✓ working.

Phase 2 — Score

For each subsystem, first identify the project delivery model and applicable outcomes in references/checklist-per-subsystem.md. Mark every finding as documented, demonstrated, or unknown; score from evidence rather than file presence alone. The canonical files in the table are useful conventions, not universal prerequisites.

• 5 = applicable outcomes are documented, demonstrated, and consistently followed
• 4 = documented and demonstrated with bounded gaps
• 3 = only documented, only demonstrated, materially partial, or unknown at a relevant boundary
• 2 = isolated or stale structure does not meet most applicable outcomes
• 1 = missing or actively harmful

For each subsystem, list:

• ✓ what's present and working
• ✗ what's missing or broken
• ~ partial / unclear

Phase 3 — Identify Bottleneck

The lowest-scoring subsystem is the bottleneck. Even if other subsystems are weaker by absolute count of checks, the lowest score is the one to fix first because it limits the value of the rest.

Tie-breaker (multiple subsystems at same low score): pick the one whose improvement unlocks progress in others. State often wins when the project lacks a durable locator for active scope and unresolved work; do not assume two particular filenames are required.

Phase 4 — Prioritized Improvement Plan

Only if the user requests recommendations, propose the smallest number of independently shippable actions that address the evidenced bottleneck. For each, name the expected evidence and a local template/example if one actually fits. Do not invent effort, score gains, or a fixed number of steps. Do not expand an audit-only request into implementation; when implementation was already requested, continue that owning task after the audit instead of requesting the same authorization again.