Assertion Quality

Step 5: Apply calibration rules

Before reporting, calibrate findings:

• Trivial means truly trivial. A null/None/nil check alone is trivial (Assert.IsNotNull(result), assert result is not None, expect(x).toBeDefined()). But a null check followed by a meaningful value assertion is not trivial — the null check is a guard before the real assertion. Only flag a test as "trivial" if it has no meaningful value assertions.
• Boolean assertions checking meaningful conditions are not trivial. Assert.IsTrue(result.IsValid) / assert result.is_valid / expect(result.isValid).toBe(true) check a specific property — these are Boolean assertions, not trivial ones. Always-true assertions (Assert.IsTrue(true), assert True, expect(true).toBe(true)) are trivial.
• Consider the test's intent. A test for a void method that verifies state change on a dependency is legitimate even if it only uses one Boolean assertion.
• Exception tests are inherently low-assertion-count. Assert.ThrowsException<T>(() => ...) / with pytest.raises(E): ... / expect(fn).toThrow(E) / #[should_panic] may be the only assertion — that's fine for exception-focused tests. Don't penalize them for low assertion count.
• Mock-call verifications and bare assertion forms count. Treat verify(mock).method(...) (Mockito), expect(mock).toHaveBeenCalledWith(...) (Jest), Should -Invoke (Pester), bare assert (pytest), if got != want { t.Errorf(...) } (Go) all as real assertions of the appropriate category. Do not treat them as missing-framework-API smells.
• Snapshot assertions (.toMatchSnapshot(), syrupy, SnapshotTesting) count as Structural/Deep assertions. Flag stale or never-updated snapshots separately.
• Property-based tests (@given Hypothesis, proptest!, forAll Kotest) generate assertions implicitly through generated cases — count the inner assertion logic, not the outer scaffold.
• Don't conflate diversity with volume. A test with 20 equality assertions has high volume but low diversity. A test with one equality, one null check, and one exception assertion has low volume but good diversity.
• Self-referential assertions are not meaningful equality checks. Asserting that an output equals an input round-trip looks like a real equality assertion but is tautological when the operation under test is expected to be identity. Flag these separately from normal equality assertions. If the test's purpose is to verify a round-trip (serialize/deserialize, encode/decode), the assertion is valid — but it should be accompanied by assertions on non-trivial inputs that exercise the transformation.
• If assertions are well-diversified, say so. A report concluding the suite has good diversity is perfectly valid.

Step 6: Report findings

Present the analysis in this structure:

1. Summary Dashboard — A quick-reference table of key metrics:

   | Metric                        | Value  | Assessment |
   |-------------------------------|--------|------------|
   | Total tests                   | 25     | —          |
   | Average assertions per test   | 2.4    | Moderate   |
   | Assertion type spread         | 5/12   | Low        |
   | Tests with zero assertions    | 3 (12%)| Concerning |
   | Tests with only trivial asserts | 4 (16%)| Acceptable |
   | Tests with negative assertions | 2 (8%) | Below target |
   | Single-category tests         | 15 (60%)| High       |

2. Category Breakdown — For each assertion category, show:

   • How many tests use it
   • Representative examples from the code
   • Whether it's overused or underused relative to the code under test

3. Gap Analysis — Based on the production code (if available), identify:

   • Behaviors that are tested but only with equality checks
   • Error paths with no exception assertions
   • State-changing methods with no state verification
   • Collections returned but never checked for contents

4. Recommendations — Prioritized list of improvements:

   • Which tests would benefit most from additional assertion types
   • Which assertion categories are missing and why they matter
   • Concrete examples of assertions that could be added

5. Assertion-free tests — If any exist, list each one with its method name and what it appears to be testing, so the user can decide whether to add assertions or mark them as intentional smoke tests.

Validation

• Every assertion in the test suite was classified into at least one category
• Metrics are computed correctly (counts add up)
• Trivial-assertion tests are correctly identified (not over-flagged)
• Exception tests are not penalized for low assertion count
• Boolean assertions on meaningful properties are not classified as trivial
• Recommendations are concrete (name specific test methods and suggest specific assertion types)
• If the suite has good diversity, the report acknowledges this

Common Pitfalls