Verifying Change Coverage

Verifying Change Coverage

Use this skill to answer one question honestly: were the lines this change introduced actually executed by a test?

Repo-wide coverage cannot answer it. A repo at 84% can merge an untested payment path and stay at 84%, because a big denominator absorbs a small numerator. Diff coverage changes the denominator to the lines you just wrote, which is the only denominator that maps to the risk you just added.

The distinction that runs through this skill: configured is not verified. A test runner in package.json proves the practice exists. Executed lines prove this change was tested. Report which of the two you measured.

When to Use

• a PR gate needs a coverage signal that cannot be gamed by unrelated tests
• total coverage is stable but defects keep escaping in new code
• diff coverage has to be wired into CI for the first time
• a reviewer needs to know which specific new lines nothing executes
• an agent wrote tests and someone needs to know whether they exercise the new code
• coverage numbers are being quoted in a release decision and nobody knows what they measure

Operating Principles

• Executed lines, not configured tools. The whole point. Say plainly which one you have.
• No artifact, no verdict. Without a coverage report you have static linkage - "a test file exists that imports this module" - which is weaker evidence and must be labelled as such. Never present it as coverage.
• Branches matter more than lines on changed code. New code is where new conditionals live. A line hit once with the if never taken is half-tested.
• Uncovered is a question, not a verdict. Some uncovered lines are fine: logging, type guards, unreachable defaults. The output is a list of uncovered changed lines with a judgement per group, not a percentage with a pass stamp.
• Coverage proves execution, never correctness. An assertion-free test covers everything and verifies nothing. Pair this with unslop-tests - a diff-coverage gate is exactly the pressure that produces coverage theatre.
• Escalate by risk, not by count. Three uncovered lines in auth outrank thirty in a formatter.
• The gate belongs on changed lines only. Demanding a repo-wide floor on a legacy codebase produces a permanently red gate that gets disabled within a month.

Workflow

Phase 0: Get the two inputs

The changed lines.

git diff <base>...HEAD --unified=0 -- '***.ts' '***.tsx'

--unified=0 gives added-line ranges with no context, which is what you want to intersect against coverage. Keep the file → line-numbers map; discard removed lines, generated files, and anything the contract excludes.

The coverage artifact. Produce one if it does not exist:

npx vitest run --coverage --coverage.reporter=lcov   # → coverage/lcov.info
npx jest --coverage --coverageReporters=lcov
npx playwright test    # + c8/istanbul instrumentation for E2E coverage

See ./resources/coverage-artifacts.md for the formats, the merge problem across suites, and the monorepo path pitfalls.

If no artifact can be produced: stop and say so. Report static linkage instead, labelled as static linkage, and state what it cannot tell you. Then make producing an artifact the first remediation item - this is the single highest-leverage change to a coverage story.

Phase 1: Intersect

For each changed source file, intersect its added-line numbers with the executed lines in the report. Produce, per file:

• changed lines - added or modified executable lines (exclude blank lines, comments, pure type declarations, and import statements)
• covered - changed lines the report marks as executed at least once
• uncovered - changed lines with zero hits
• partial branches - changed lines with branch data where at least one path was never taken

Diff coverage = covered ÷ changed executable lines, per file and overall. Compute branch coverage on changed lines separately; do not average it into the line number.

The exclusions matter and are the most common source of a wrong figure - see ./resources/coverage-artifacts.md for what counts as an executable line in TypeScript once it has been compiled or transformed.

Phase 2: Classify each uncovered group

Never hand over a raw list. Group contiguous uncovered lines and judge each group:

The first two are findings. The rest are exceptions and must be recorded, not silently subtracted - see ./resources/exceptions-and-thresholds.md.

Phase 3: Set the verdict

Combine diff coverage with the risk tags from scoping-change-relevance:

The escalation rule is the reason this beats a flat percentage: 70% on a logging module and 70% on session handling are not the same result, and one number cannot say so.