Test Anti Patterns

Step 4: Calibrate severity honestly

Before reporting, re-check each finding against these severity rules:

• Critical/High: Only for issues that cause tests to give false confidence or be unreliable. A test that always passes regardless of correctness is Critical. Shared mutable state is High when it is a latent isolation risk, but Critical when the user reports actual order-dependent failures or the code proves one test requires another to run first. Missing-await on async assertions is Critical (silent pass).
• Medium: Only for issues that actively harm maintainability -- 5+ nearly-identical tests, truly meaningless names like Test1 / test / it1.
• Low: Cosmetic naming mismatches, minor style preferences, assertion messages that could be better. When in doubt, rate Low.
• Use the caller's severity vocabulary consistently. If the caller asks for Critical / Warning / Info, map latent reliability risks to Warning and maintenance/cosmetic concerns to Info. Keep a demonstrated false-confidence or current order-dependency root cause Critical; do not downgrade it merely to make every requested tier non-empty. Severity describes the demonstrated failure mode, not how much prose a finding receives.
• Separate a systemic finding from its instances. Coverage touching across a facade is one Critical systemic finding whose evidence lists every affected test. All assertion-free instances, including the last facade method, retain the same false-confidence severity. Report 1 finding / 6 affected tests, not six findings plus a seventh summary finding, and do not downgrade one instance merely to manufacture multiple tiers.
• Do not severity-rank ordinary missing cases as anti-patterns. Adjacent untested branches, exception paths, and boundaries may be useful coverage opportunities, but list them separately from the anti-pattern counts unless a weak existing test specifically creates the gap. They are not Critical merely because the suite has a systemic Critical issue.
• Not an issue (per-language nuance):
  • Go and Rust table-driven loops with sub-tests (t.Run / for case in cases { ... }) are idiomatic, not "Conditional Test Logic". Do NOT flag.
  • pytest bare assert is the canonical assertion form, not a missing assertion library. Do NOT flag.
  • Go tests use if got != want { t.Errorf(...) } as canonical equality. Do NOT flag as ad-hoc.
  • Separate tests for distinct boundary conditions (zero vs. negative vs. null). Do NOT flag as duplicates.
  • Explicit per-test setup instead of [TestInitialize] / beforeEach (this improves isolation).
  • Tests that are short and clear but could theoretically be consolidated.
  • Round-trip or serialization equality with non-trivial input. It is valid metamorphic evidence, not a self-comparison; still recommend one independent representation when producer and consumer could share a defect.
  • A transformation tested only with an already-transformed input. Keep it out of the tautology count, but report the weak oracle when removing the transformation would still pass. Use an input that must change and pin its independently expected output.
  • Clone value equality. Keep it, and add distinct-reference or mutation- independence evidence when the contract promises a deep copy.
  • A validator or accessor returning the original value when pass-through is the production contract. Missing invalid-input cases are a coverage gap, not proof that the existing assertion is tautological.

IMPORTANT: If the tests are well-written, say so clearly up front. Do not inflate severity to justify the review. A review that finds zero Critical/High issues and only minor Low suggestions is a valid and valuable outcome. Lead with what the tests do well.

Step 5: Report findings

Depth bar — a tidy report that is shallower than an unassisted review is a failure. Before writing, satisfy all five:

1. Account for every test in scope. Build the complete method/field inventory before summarizing. For a systematic pattern such as coverage touching, enumerate every affected test at least once rather than giving representative examples. A finding table that silently skips tests (or fixtures like an unused static HttpClient field) is incomplete. State the number reviewed.
2. Verify the production contract before judging the oracle. Inspect the actual transformation, DTO fields, and promised identity/clone semantics. Never invent fields or require lossless round-tripping when production is intentionally lossy. For every suspicious equality, write down the independently known oracle before assigning a finding. If the assertion compares a transformed output with non-trivial input, clone state, snapshot, mock verification, or a framework-native assertion context, explain why it can fail before calling it tautological or assertion-free. Conversely, when a transformation test uses an already-normalized input, call out that the input cannot distinguish the real transformation from a no-op and provide a changing input plus exact expected output. For paired producer/consumer APIs, retain the round-trip test and add one independent representation oracle rather than replacing valid metamorphic evidence.
3. Make every Critical/High fix complete and specific. Give the replacement assertion with the exact expected value (the computed discount, the exact CSV line, the full expected object), not a // assert something here placeholder.
4. Name obvious adjacent gaps without widening into mutation analysis — when production code is supplied, note directly related untested throws, null results, boundary values, and round-trip/culture-sensitivity risks in an Adjacent coverage gaps section. Use test-gap-analysis for exhaustive branch-by-branch behavioral gaps.
5. Keep the report internally consistent. Summary counts must equal the enumerated findings. Publish a settled conclusion: do all reconsidering before you write, and never leave "wait, that's wrong" / "this should fail but doesn't" reasoning in the output.
6. Make non-findings decisive. For a clean or mostly clean small suite, name the suspicious constructs you cleared and the framework rule that makes each valid. Do not bury a clean verdict under a generic checklist or speculative improvements.

Present findings in this structure:

1. Summary -- Total issues found, broken down by severity (Critical / High / Medium / Low). If tests are well-written, lead with that assessment.
2. Critical and High findings -- List each with:
   • The anti-pattern name
   • The specific location (file, method name, line)
   • A brief explanation of why it's a problem
   • A concrete fix (show before/after code when helpful)
3. Medium and Low findings -- Summarize in a table unless the user wants full detail
4. Positive observations -- Call out things the tests do well (sealed class, specific exception types, data-driven tests, clear AAA structure, proper use of fakes, good naming). Don't only report negatives.

Before publishing, assign each finding a stable identity. A grouped row counts as one finding regardless of how many methods it lists; separate rows count separately. Recompute the summary from those rows. Keep affected tests as a different number so a bundled finding cannot create a hidden count mismatch.

Step 6: Prioritize recommendations

If there are many findings, recommend which to fix first:

1. Critical -- Fix immediately, these tests may be giving false confidence
2. High -- Fix soon, these cause flakiness or maintenance burden
3. Medium/Low -- Fix opportunistically during related edits

Validation

• Every test method in scope is accounted for (reviewed count stated; none silently skipped)
• Identity and round-trip findings match the production contract and use only real fields
• Every finding includes a specific location (not just a general warning)
• Every Critical/High finding includes a concrete fix with exact expected values
• Adjacent untested error paths and boundary values are called out
• Summary counts match the enumerated findings
• Grouped findings distinguish finding count from affected-test count
• Adjacent coverage opportunities are not inflated into Critical anti-pattern findings
• Report covers all categories (assertions, isolation, naming, structure)
• Positive observations are included alongside problems
• Recommendations are prioritized by severity

Common Pitfalls