External API Change

Changing code that talks to an external web service

This is, by the project's own account, the most fragile part of the codebase (see CLAUDE.md gotchas and .claude/context.md fragile spots #1). UniProt, Ensembl, PANTHER, PhylomeDB, OrthoInspector, KEGG, and GO change response formats, rename fields, and go down without notice — all outside RNAlysis' control. The relevant code lives in rnalysis/utils/io.py (async aiohttp + tenacity retries + aiolimiter rate limiting + response caches) and its callers: enrichment.py, utils/enrichment_runner.py, utils/ontology.py.

Real precedents worth knowing before you start: OrthoInspector relocated its whole API to api.bigest-icube.fr; Ensembl silently dropped cross-division ortholog data (worm→human now returns nothing); PANTHER can return an empty HTTP 200 body instead of an error. None of these were visible from reading old code — each needed a live check.

This is TDD like everything else in the repo, with one twist: step 1 happens before you write any test, because you cannot write an honest test against a service whose current behavior you haven't confirmed.

When to use this vs. skip it

Use this skill for any change to code that issues a request to one of the six services above, or adds a new one. Skip it for CLI-tool bridges (kallisto/bowtie2/cutadapt/R) — those have their own fragility (see CLAUDE.md's R/permissions gotcha and .claude/workflows.md's R-bridge workflow) but aren't a web-API problem.

Step 1 — Confirm the service's REAL current behavior before writing any code

Do not trust the existing code, your training data, or memory of "how this API works" — these services change often enough that stale assumptions are the #1 way this class of fix goes wrong. Before touching io.py or its callers:

• Hit the endpoint live (curl / browser / a throwaway Python snippet) and read the actual response — status code, headers, body shape, field names.
• Or use the research skill to investigate and record what you found.
• If network access isn't available in your environment, say so explicitly and get a human (or an agent with network access) to confirm before coding against assumptions.

Write down what you found (in the PR description at minimum) — the next person debugging this service needs that evidence trail, not just the resulting diff.

Step 2 — Preserve the io.py scaffolding; degrade gracefully

Keep the existing resilience layers intact while you fix the specific breakage:

• tenacity retry decorators around the request.
• aiolimiter rate limiting (services like UniProt idmapping throttle hard; CI already hits this — see the CI note below).
• The response caches (don't bypass them while debugging and then forget to restore them).

A changed or dead service must never crash the app. If the new response shape can't be parsed, fail toward an empty/partial result with a clear, user-facing message — not an unhandled exception. This is a non-programmer-facing GUI (per CLAUDE.md); a stack trace is not an acceptable failure mode for "PANTHER is down today."

Step 3 — Capture a real payload as a test fixture, then mock it

Tests must never depend on the live service (flaky by construction, and CI's network tier is narrow — see below). Capture one:

python packaging/snapshot_api_payload.py https://rest.uniprot.org/idmapping/status/abc123
python packaging/snapshot_api_payload.py https://api.example.org/search \
    --param query=BRCA1 --header "Accept: application/json" \
    --out tests/test_files/example_search_response.json

This provider-neutral helper does a plain GET and writes the raw response body byte-for-byte to tests/test_files/ (default) or --out, deriving a filename from the URL when you don't name one. Run python packaging/snapshot_api_payload.py --help for the full option list. It makes a real live request — run it locally/by hand where you have network, not in a sandboxed or offline environment. It does not itself write or run any test; it only produces the fixture file.

Then write the test in the matching tests/test_*.py (test_io.py for most of these) mocking the HTTP layer against that fixture. The repo's established convention is requests_mock for synchronous requests calls (see the many with requests_mock.Mocker() as m: blocks already in tests/test_io.py) — match whichever HTTP client (requests vs aiohttp) the code under test actually uses, and mock at that layer.

Red → green → refactor from here as usual: write the failing test against the captured fixture, make it pass, clean up, run the module's test file.