Testing Py

Testing Python

A green suite is not a tested suite. Most Python suites pass while shipping bugs because they lie in four predictable ways, and your job is to refuse each lie:

1. Line-only coverage reports 95% while every else and except arm is unvisited.
2. Mocks patched at the wrong place never apply, so the test runs the real code or asserts nothing.
3. autospec-less mocks pass even when you typo a method name or change a signature — a false green.
4. autouse fixtures mutate hidden global state, so tests pass alone and fail in CI order.

The posture, non-negotiable: pytest 9 strict markers and strict config on, branch coverage on, autospec= on every mock, and Hypothesis property tests for any pure logic with an invariant. Assert on behavior, never on a mock's internals.

Versions you are on (2026-06): pytest 9.0.2, coverage.py 7.14.1, pytest-cov 7.x (needs coverage ≥ 7.10.6), Hypothesis 6.155.x.

Which test do I need?

Pick the cheapest test that still catches the bug.

Default to property + parametrized for logic; reserve mocks for boundaries you do not own.

Layout and config

tests/ mirrors the package tree. Put each fixture in the nearest conftest.py, not always the root one — proximity is documentation. Drive everything from pyproject.toml:

[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = "--strict-markers --strict-config --cov=mypkg --cov-branch --cov-report=term-missing --cov-fail-under=85"
markers = [
  "slow: deselect with -m 'not slow'",
  "integration: touches a real boundary",
]

[tool.coverage.run]
branch = true            # measure both arms of every conditional, not just lines
source = ["mypkg"]

[tool.coverage.report]
show_missing = true
fail_under = 85
exclude_lines = [
  "if TYPE_CHECKING:",
  "raise NotImplementedError",
  "def __repr__",
  "pragma: no cover",
]

--strict-markers turns a typo'd @pytest.mark.slwo into an error instead of a silently-skipped test. --strict-config rejects unknown config keys. pytest 9 also exposes strict_markers, strict_config, strict_xfail, and strict_parametrization_ids as config aliases — treat strict as the default, not an upgrade.

Fixtures

Scope is a contract about reuse and isolation. Default to function and widen only when setup is genuinely expensive and the object is genuinely immutable.

A yield fixture's pre-yield code is setup, post-yield is teardown — and teardown runs even when the test fails, so it is where cleanup belongs.

# Bad: everything in the root conftest.py, plus a hidden autouse reset.
# Nobody can trace why a test depends on the DB, and the reset masks state leaks.
@pytest.fixture(autouse=True)
def reset_db():
    db.truncate_all()

# Good: scoped fixture in tests/api/conftest.py, explicit, documented.
# The docstring shows up in `pytest --fixtures`, so it is discoverable.
@pytest.fixture
def api_client(tmp_path):
    """An isolated API client backed by a throwaway sqlite file."""
    client = ApiClient(db_path=tmp_path / "test.db")
    yield client
    client.close()  # runs even if the test raises

Parametrize a fixture when several tests should each run against multiple backings:

@pytest.fixture(params=["sqlite", "memory"])
def store(request):
    return make_store(request.param)  # every test using `store` runs twice

Use autouse=True only for something every test in scope truly needs (a frozen clock, a captured log). A hidden global dependency makes a failure impossible to read.

Mocking

Patch the name where it is looked up, not where it is defined. This is the single most common mocking bug. If mymod does from httpx import get, then get now lives at mymod.get; patching httpx.get changes a name mymod no longer reads, so the mock never applies and the test silently hits the network.

# Bad: patches the definition site; mymod already bound its own `get` at import.
with patch("httpx.get") as m:        # no effect on mymod.get
    result = mymod.fetch(url)

# Good: patch the name in the module under test, and autospec it.
with patch("mymod.get", autospec=True) as m:
    m.return_value.json.return_value = {"ok": True}
    result = mymod.fetch(url)
    m.assert_called_once_with(url, timeout=5)

autospec=True is mandatory. Without it a typo'd method or a wrong arg count still passes — the mock invents any attribute you touch. With autospec the mock enforces the real object's signature, so a refactor that renames a method or drops a parameter fails the test like it should. Use create_autospec(obj) when you need a standalone fake of a class or callable.

Choosing the swap tool:

• monkeypatch (built into pytest, no install): env vars, attributes, chdir. monkeypatch.setenv / delenv / setattr / chdir auto-revert after the test. Use it for "swap one thing for this test."
• mocker (pytest-mock): unittest.mock with automatic undo and call tracking. Less boilerplate for return_value / side_effect and assert_called_*. Use it for "fake a callable and inspect how it was called."

Both are correct; pick by what you are swapping. Faking time, HTTP, DB, the filesystem, side_effect, asserting calls, and AsyncMock for async code live in references/mocking.md.

Coverage that means something

Run it with branch coverage and a floor:

pytest --cov=mypkg --cov-branch --cov-report=term-missing --cov-fail-under=85

Line coverage lies because executing a line is not the same as testing its outcomes. A function with if x: a() else: b() reaches 100% line coverage from a single test that only takes the if arm — the else ships untested. --cov-branch (or branch = true) forces both arms to be exercised, which is exactly where happy-path-only suites leak bugs. term-missing prints the unhit line and branch numbers so you know what to write next. Exclude only code that is correctly never run by tests (if TYPE_CHECKING:, raise NotImplementedError, __repr__) — never exclude a branch just to make the number go up.