Verification

Verification

Task completion is not goal achievement. Verify that the phase achieved its goal, not that prior agents said it did. If you cannot independently reproduce a claimed success, return FAIL.

Reference Files

• references/live-production-testing.md — live/production verification strategy

The Gate Function

COMPLETION → TRUTH → PROOF

1. Completion: Did the agent finish its task? (contract says PASS)
2. Truth: Is the work actually correct? (independent verification)
3. Proof: Can you prove it with evidence? (exit codes, test output, screenshots)

A PASS without proof is a claim. When a result surprises you in either direction — a pass that came too easily, or a failure you cannot explain — suspect the observation method before the system: confirm the check runs the right command against the current build and exercises the real code path before theorizing about the code.

<!-- Authoring rule (maintenance, not runtime): every gate in this skill encodes an observed failure mode; understand what a gate prevents before removing it. -->

Self-Critique Gate (BEFORE Verification Commands)

Before running any test, audit your own work:

Code Quality:

• No TODO/FIXME/stubs in changed files
• No commented-out code
• No debug logging left in
• Error handling covers the failure modes the change introduces
• Types are correct (not as any or as unknown)

Implementation Completeness:

• Every acceptance criterion from the plan has corresponding code
• Every named scenario has a test
• Every error path in the plan has handling
• No silent failures (empty catches, discarded errors)
• Build succeeds, type-check passes

Self-Critique Verdict: If any check fails, fix it BEFORE running verification. Don't run tests you know will fail on code quality issues — a known-failing run produces noise evidence that then has to be explained away.

Validation Levels

Every verification must state its validation level — unlabeled evidence lets weak proof masquerade as strong. If manual, state the checklist. Manual evidence is a validation level for verification, never a substitute for a TDD RED or GREEN exit code: with no test runner and no scripted check, require a runner or block. If deterministic, state the command + exit code. If live, state the harness command.

Production-Like Live Proof

When the plan includes ### Live Verification Strategy or a harness manifest:

• Run python3 "${CLAUDE_PLUGIN_ROOT}/tools/live_harness_runner.py" --manifest <path> --mode proof
• If stress required: also run --mode stress
• Do NOT silently substitute replay fixtures or unit tests for required live proof

Flaky test handling: re-run once — one retry separates environment blips from real flake; more retries launder genuine failures. Pass on re-run → mark PASS with flaky: true. Fail both → FAIL. Never convert flaky pass into unconditional confidence.

Evidence Array Protocol (MANDATORY for PASS)

EVIDENCE:
  scenarios:
    - "[name] | Given [state] | When [action] | [command] → exit [code] | expected=[expected] | actual=[actual]"
  regressions:
    - "[test] → exit [code]: [result]"
  edge_cases:
    - "[case]: [command] → exit [code]: [result]"

Every scenario needs non-empty Expected and Actual. Every scenario maps to exactly one EVIDENCE entry. SCENARIOS_PASSED must equal EVIDENCE.scenarios with exit 0 + Result=PASS. Run each check against the revision being verified; if the code changes after a check, re-run the affected scenarios before citing them. Record the tested identity with the evidence: the commit SHA, plus a patch or digest when the tree is dirty; evidence that does not identify the revision it was produced from is unverified, not PASS.

Goal-Backward Lens

Walk backward from the goal to verify it was achieved:

1. What was the goal? (re-read the plan's exit criteria)
2. What would prove it? (name the specific evidence)
3. Do I have that evidence? (run the verification)
4. Does the evidence actually prove the goal? (not "tests pass" but "the tests test the right thing")