Test Audit

Test Audit

Tests earn their place by catching a credible regression that nothing else catches. Agents write a test for every small change, and most of those re-assert the source, replay a stronger test through a mock, or pin a private call shape. Coverage barely moves when they go; maintenance cost and suite time do.

• IS: sweeping a directory for low-value tests, and campaign-pruning a whole subsystem or repo to a measured target with coverage held, plus deleting the test-only production seams those tests kept alive.
• IS NOT: gating a test before it lands or reviewing a feature diff (tidy), making CI faster by splitting or sharding (ci-speedup), or writing tests for untested code.

Pick a mode

Set a target before starting

An open-ended "clean up the tests" stops after a few obvious deletions, because every remaining candidate looks defensible in isolation. Audit and Campaign need a number to work against. When the user gave one, use it. When they did not, propose one and proceed on it rather than waiting:

Remove the least useful 20% of test declarations in <scope>, keeping line and branch coverage within 2 percentage points of the baseline and every retained test green.

Measure the baseline first (references/coverage.md), then keep deleting in order of lowest value until the target is met or the coverage budget is spent. Stopping short is a valid result only with a ledger showing that the remaining candidates each guard a named contract. The target is a floor on effort, not permission to cut uncertain tests: the retention bar below still wins over the number.

Junk patterns

What an audit or campaign hunts for:

• assertion-free coverage probes, and tests whose only assertion is "does not throw" on a path that cannot throw
• self-comparisons, identity copies, and expected values produced by the helper or renderer under test
• copied fixtures, inventories, manifests, export lists, or snapshots of data the test also defines
• exact source, import, or string greps that fail on a rename and survive a real behavior change
• private predicate or call-shape tests (toHaveBeenCalledWith on an internal) duplicated at a real boundary
• duplicate invocations of one contract, and per-provider replays of a shared helper
• tests whose only purpose is keeping a test-only export, global, or wrapper alive, and production code whose only callers are tests
• mocks that implement the asserted behavior, or one mock standing in for several different APIs
• fixtures that supply the ordering, receipt, or callback the code under test should produce, or persistence asserted against a store the path never writes
• capability tests that restate declared flags instead of exercising what the flag promises
• negative controls that pass for an unrelated reason, such as a rejection from a different guard
• names that promise more than the assertions check: judge the test by its assertions, not its title

Retention bar

Keep a test when it independently enforces a public API, protocol, config, migration, storage, security, platform, default value, generated cross-language, package, release, or architecture contract. Also keep:

• call ordering when the order is observable behavior
• a regression with a credible failure mode
• a source inspection when it is the cheapest independent guard: it fails when the user-facing key, byte, or path changes and survives an identifier-only rename
• a retained test that fails on the baseline: treat it as a possible product bug, reproduce it, and fix the owner rather than deleting it

Static or slow is not a reason to delete. A test that resembles implementation may still be the only proof of a contract; show otherwise before removing it.

Evidence per candidate

Before judging, read the complete test, the production owner, its callers and callees, sibling implementations, overlapping tests, and the history that explains why the test exists. Record for each deletion:

• test name and location, and what failure it can actually detect
• non-test callers of the seam it covers
• the stronger proof that remains (name the keeper test), or why no contract exists
• the production or test-support code the deletion unlocks
• coverage delta from removing it, when measured

A missing field means the candidate is not ready. Prefer a few high-confidence deletions over a long speculative list; do not convert uncertain candidates into deletions to reach the number.