Test Coverage

Test Coverage (patch-scoped)

Answers two questions about the current branch's patch vs origin/main: are its tests passing, and is it adequately covered? Both come from the same scoped test run, so one skill owns both.

Why patch-scoped, not full-suite

The full suite is 349 packages / 1876 test files and takes 45-75 minutes per CI timeouts — running it every hour is not feasible. This skill only ever runs tests for packages containing files this patch touched — it never expands to the other 340+ packages just to go looking for trouble. But once a package is in scope and its run turns up a failure, fix it regardless of whether that specific test or file is what this patch changed: what this loop cares about is the packages it touched actually passing, not whose commit originally broke them.

Running the check

atmos fix coverage [base-ref]

(atmos fix tests [base-ref] is an equivalent alias — same underlying run, reach for it when the question framing is "are my tests passing" rather than "is my patch covered.") Defaults to origin/main. This wraps .claude/skills/test-coverage/scripts/patch-test-coverage.sh, which deliberately does no line-level analysis itself — it just prints a raw bundle (pass/fail status, raw test output if failed, the coverage profile if passed, and the diff) for the fixing agent to reason over directly, the same way coderabbit-review reads raw CodeRabbit markdown rather than us pre-parsing it.

Three outcomes:

• STATUS: NO_GO_CHANGES — no touched .go files. One-line no-op, done.
• STATUS: TESTS_FAILING — go to "Fix failing tests" below.
• STATUS: OK — tests pass; go to "Fix coverage gaps" below.

Fix failing tests (gates coverage — a package with failing tests has meaningless coverage

numbers)

Delegate to Agent subagent_type: "test-coverage-fix", Section A, passing the raw failing-test output and the list of touched files. The agent classifies each failure as in-scope (the file lives among this patch's touched files) or pre-existing (it doesn't), but scope only changes how much diagnosis a fix needs — not whether an attempt is made. What this loop cares about is the suite passing, full stop; "not this patch's fault" is not a reason to leave something red. Confirmed for real: a "pre-existing failure" was itself a false positive in the check, not the code — the patch-scoped go test run had no -timeout override, so Go's 10-minute binary default tripped on the full (patch-unrelated) ./tests acceptance package under load, panicking on whichever subtest happened to still be running when the clock ran out. The fix was to the test-infrastructure itself (patch-test-coverage.sh now passes -timeout 40m, matching .atmos.d/test.yaml's full-suite convention) — exactly the kind of pre-existing-but-fixable root cause this agent should resolve rather than just report.

Non-negotiable, equal severity to the anti-theater rule below: never weaken or delete an assertion just to force a red test green. If the test's own expectation was wrong (not the code), fix the test to assert the correct behavior and say so explicitly — don't just make it pass.

One fix attempt per failing test per cycle, in-scope or pre-existing alike. Still red after that, or the agent isn't confident it can safely attempt a fix at all (e.g. it would require touching a hard-prohibited file, or needs a credential/decision only a human has): stop, report for human attention, and invoke the say skill with something like "PR <number> coverage check needs your input." Don't loop indefinitely inside an hourly budget.

Only proceed to coverage gaps when every scoped test passes. If any failure remains red after its single fix attempt, report it, invoke say, and stop; do not analyze coverage from a failing test run.