Test Coverage Review

Test Coverage Review

This skill audits test coverage in Python projects, identifies gaps, writes new tests, and verifies that the test suite passes — all following pytest best practices.

Before you start

Read the best-practices reference file to ground your work:

cat references/best-practices.md

Use the practices and source citations in that file as your authoritative checklist.

Modes of operation

Determine the mode from the user's request.

Mode 1 — Audit existing test coverage

Use when the user says "review my tests", "check my coverage", or "are my tests good enough".

Step 1: Discover the project structure

Identify:

• The package name and layout (flat or src/ layout)
• The test directory (tests/, test/, or co-located)
• The test framework in use (pytest, unittest, or both)
• Any conftest.py files and shared fixtures
• Coverage configuration in pyproject.toml, setup.cfg, or .coveragerc

Step 2: Map source modules to test files

For every public module in the package, check whether a corresponding test file exists. Flag modules with no test file at all.

Step 3: Audit test quality

For each existing test file, check against the reference checklist:

1. Naming — File named test_<module>.py, classes prefixed Test, methods prefixed test_.
2. Docstrings — Module-level docstring, class docstring, method docstrings.
3. Structure — Tests grouped in classes by the unit under test, with section dividers between classes.
4. Isolation — No test depends on another test's state or execution order.
5. Mock boundaries — External I/O (HTTP, filesystem, database) is mocked. Internal methods are tested directly, not mocked.
6. Fixtures — Shared setup uses @pytest.fixture, not setUp() or duplicated code.
7. Sample data — Test data defined as module-level constants (not buried inside fixtures or test methods).
8. Edge cases — Empty inputs, invalid inputs, boundary conditions, and error paths are tested.
9. Assertions — Each test has clear, specific assertions. No tests that only check "does not raise".
10. Coverage — Every public method/function has at least one test. Every branch (if/else, try/except) has coverage.

Step 4: Run coverage (if possible)

pytest tests/ --cov=<package> --cov-report=term-missing --tb=short -q

Report:

• Overall coverage percentage
• Specific lines/branches not covered
• Which modules are below the project's coverage threshold

Step 5: Produce the audit report

Output a markdown table of findings:

Severity levels:

• critical — Public code with no tests, untested error paths, missing test files, live API calls in tests.
• warning — Missing docstrings, over-mocking, duplicated setup code, missing edge cases.
• info — Style suggestions, parametrize opportunities, fixture improvements.

Mode 2 — Write tests for new code

Use when the user says "write tests for this", "add tests for this module", or has just finished a feature.

Step 1: Analyze the code under test

Read the module and identify:

• Every public class and its public methods
• Every public function
• Constructor parameters and their types
• Return types and possible exceptions
• External dependencies (HTTP calls, file I/O, database, etc.)
• Edge cases (empty inputs, None values, invalid types, boundary conditions)

Step 2: Plan test categories

For each public class or function, plan tests in these categories:

Present the plan to the user before writing.

Step 3: Write the test file

Follow this structure:

"""Tests for <module description>."""

from unittest.mock import patch, MagicMock

import pytest

from <package>.<module> import <Class>, <function>


# ---------------------------------------------------------------------------
# Sample data
# ---------------------------------------------------------------------------

SAMPLE_DATA = ...  # Module-level test data constants


# ---------------------------------------------------------------------------
# <ClassName> tests
# ---------------------------------------------------------------------------


class Test<ClassName>:
    """Tests for the <ClassName> <type>."""

    def test_<behavior>(self):
        """Verify <what is being tested>."""
        # Arrange
        ...
        # Act
        ...
        # Assert
        ...

Rules:

• One test file per source module: test_<module>.py
• Group related tests in class Test<Name>: with class docstrings
• Docstring on every test method: """Verify <what>."""
• Section dividers (# ---) between test classes
• Follow Arrange-Act-Assert pattern within each test
• Sample data as module-level constants, not inside tests
• Shared setup in @pytest.fixture, not setUp()

Step 4: Mock at the boundary

Mock external dependencies only:

• requests.get / requests.post for HTTP calls
• open / pathlib.Path for file I/O
• Database connections and cursors
• Third-party API clients
• time.sleep, datetime.now when testing time-dependent logic

Do NOT mock:

• Internal methods of the class under test
• Private helper functions
• Data transformation logic
• Anything that can be tested directly with sample data

# CORRECT — mock the HTTP boundary
@patch("mypackage.api.requests.get")
def test_fetch_sends_timeout(self, mock_get):
    mock_get.return_value = MagicMock(status_code=200, json=lambda: {})
    client = APIClient()
    client.fetch("https://example.com")
    _, kwargs = mock_get.call_args
    assert kwargs["timeout"] == 10

# WRONG — don't mock internal methods
@patch.object(Parser, "_transform")  # ← test this directly instead
def test_parse(self, mock_transform):
    ...

Step 5: Run and verify

After writing tests:

# Run the new tests
pytest tests/test_<module>.py -v --tb=short

# Run the full suite to check for regressions
pytest tests/ --tb=short -q

# Check coverage of the new module
pytest tests/ --cov=<package>.<module> --cov-report=term-missing

Report the test count and pass/fail status. Note the count for the changelog entry:

- **tests/test_<module>.py**: N unit tests for <description>.