Shipping Across Surfaces

Shipping Across Surfaces

Most product facts are stated more than once. The feature exists in code, is described on a landing page, summarized for agents, listed in a changelog, repeated in a nav badge, restated in a README, and embedded again in the description string of every tool or endpoint that exposes it. Each copy drifts independently, and none of them are covered by tests.

The failure is never loud. Nothing goes red; the product simply describes itself inaccurately in the four places you didn't edit, and the one place a reader happened to look is the stale one.

Enumerate the surface inventory once, in the repo

"I'll remember the other places" does not survive the second feature. Write the list down — in CONTRIBUTING.md, a PR template, or the skill/guide the project already keeps — and treat it as the definition of done for a user-facing change. A realistic inventory for a product with a public site:

• Landing page: hero, section ledes, the feature cards, the free/paid list
• <meta> / OG / Twitter tags, and the structured data (SoftwareApplication description, FAQPage entries) — search engines read the copy you forgot
• Docs page, and any machine-readable summary for agents (llms.txt)
• Changelog entry and the version badge repeated in the nav of every page
• Sitemap, and the footer product column present on every page
• README, comparison pages, onboarding/skill files
• The descriptions embedded in code — every tool, command, or endpoint description, not just the primary one

That last bullet is the one that gets missed. A rename lands in the tool it was named after and stays wrong in the three sibling tools that mention it in passing. Grep for the old string across the whole tree, not just the docs directory, before calling the change done.

All of these ship in the same PR. A follow-up "update docs" commit means the interval between them shipped a product that contradicted itself.

Prefer a generated surface to a restated one

The surfaces above are drudgery precisely because they're copies. Delete the copy where you can:

• An API reference rendered from the live schema self-syncs and needs no hand edit when the API changes. That page is free forever.
• A version badge repeated across forty pages belongs in a template/partial or a build-time injection, not in forty files.
• A "supported features" list is better derived from the registry it describes than typed a second time.

Where generation isn't practical, make drift fail a test rather than a review.

def test_readme_documents_every_registered_tool():
    registered = {t.name for t in get_registered_tools()}
    documented = parse_tool_names(README.read_text())
    assert documented == registered   # fails on add, remove, and rename

Compare against the live registry, not a second hand-written list — a test that compares two hand-maintained lists only proves you updated both copies of the same mistake. The same rule covers duplicated dependency metadata and committed build outputs; see build-artifacts (also in this repo's .claude/skills/).

A response model is only half the contract

A typed backend and an untyped frontend can disagree without either side failing. Renaming a Pydantic field updates serialization and keeps every Python test green, while a Jinja template's inline JavaScript still reads the old name inside a template literal. The browser then renders undefined: valid JavaScript, no exception, no backend failure.

Treat every serialized field rename as a producer-and-consumer change:

# Search the whole tree, not just Python call sites. Templates and committed
# JavaScript bundles are consumers too.
rg 'old_field|new_field' .

Then pin the boundary with a test that uses the real serialized response and the real consumer. A browser test is ideal when available. A cheaper contract test can still make the hidden dependency explicit:

def test_dashboard_consumes_the_audit_response_contract():
    payload = AuditResponse.example().model_dump()
    template = Path("templates/dashboard.html").read_text()

    for field in ("summary", "recommendations"):
        assert field in payload
        assert f"result.{field}" in template

This test is deliberately narrow: it does not claim to execute JavaScript. It makes a field rename fail in the same change that alters the response model, instead of relying on someone to notice undefined in a rendered page. Prefer generating a typed client or shared schema when the frontend architecture allows it; otherwise keep this boundary test beside the producer's contract tests.

Don't hand-maintain a copy of a surface you don't own

The tempting shortcut when validating against another system — another service's tool names, another team's enum, a partner API's status codes — is to paste the current values into a constant and check against it.

# Anti-pattern: a private snapshot of someone else's public surface.
KNOWN_TOOLS = {"search", "create", "archive"}   # correct until they ship

It is stale the moment the owner ships, and the staleness surfaces as your validator rejecting valid input. Worse, nothing in your repo can detect it: you have no reference to compare against.

Truth flows outward from whoever owns it. The owning codebase publishes a generated, versioned artifact — a committed tools.json carrying the source version and commit — and the consumer reads that file. Ask "who owns this fact, and which direction is it moving?" before writing the check. A design where your repo reaches into theirs to scrape the truth is the wrong direction and will break on access, auth, or refactor; a design where they publish and you consume is stable.

Checks that need no external truth are fine to land standalone: a denylist of known-bad legacy patterns, a structural schema check, a "this field is required" assertion. Those describe your own expectations, not their surface.