Distilled SDK Update

Updating a distilled SDK to its latest spec

Every patch under packages/<pkg>/patches/ is a claim that the upstream description is wrong. Upstreams fix things, so every regeneration is also the moment to drop the patches that no longer change anything. The two halves are one job: a spec update that keeps stale patches hides the fix, and a patch audit against an old spec finds nothing.

Work in a worktree cut from origin/main and run pnpm install there. Nothing here needs a full pnpm specs:sync; one mirror is enough.

Step 1 — move the mirror

pnpm --filter @distilled.cloud/<pkg> run specs:fetch    # initialise at the committed commit
pnpm --filter @distilled.cloud/<pkg> run specs:update   # move to the mirror's tip
git -C packages/<pkg>/specs/spec-mirror-<pkg> log -1 --format='%h %ci'

specs:update alone skips a submodule that was never initialised, which is why specs:fetch comes first. git status now shows the gitlink (M packages/<pkg>/specs/spec-mirror-<pkg>) — that line is part of the commit; it is how anyone else reproduces the generation.

The mirror refetches daily, so its tip is at most a day behind upstream. When you need today's upstream (a provider just published a fix), preview with pnpm specs:local <pkg> and DISTILLED_SPECS_LOCAL=1 pnpm generate <pkg>, but commit only a generation from the mirror gitlink — a .local generation is one nobody can reproduce.

Step 2 — regenerate

pnpm generate <pkg>

A patch whose pointer no longer resolves fails convert with ❌ bad patch: <file> [<op> <path>]: stale target. That is the new spec telling you something moved: either upstream now has what the op added (delete the op) or the path changed (re-point it). Fix the patch and rerun; onStalePatch: "warn" is how a whole chain once vanished silently.

Step 3 — audit the patches

pnpm patches:audit <pkg>          # one verdict per file
pnpm patches:audit <pkg> --ops    # also one per op inside every needed file
pnpm patches:audit <pkg> --only <substring>
pnpm patches:audit <pkg> --jobs <n>   # parallel converts; default from cores and free memory

Run the audit only after the mirror has moved (step 1): it answers which patches the new spec has absorbed, and nothing else. Always name the package. To check what one patch you just wrote or edited does, read the generated diff instead (distilled-sdk-patch, step 5).

The audit (@distilled.cloud/core/codegen/patch-audit) builds the model once with every patch, then once per patch with that patch left out, and diffs the result. It runs convert on scratch copies of the package (packages/.audit-<pkg>-<n>, removed on exit; the package itself is never written), so the spec mirror must be fetched (step 1); without it the package is reported as skipped. Patches are Smithy ops applied in finalizeConvert, so each file is judged in memory: one convert, then only the finalize steps re-run per file. All of cloudflare (about 2,850 files) takes under two minutes, posthog and azure well under a minute, most packages seconds. Railway's GraphQL patches apply outside finalizeConvert and cost one convert each (DISTILLED_SKIP_PATCHES), spread over --jobs copies.

Verdicts are one-at-a-time. Two patches that add the same thing each look unused alone; only the second deletion changes the model. So: delete what it lists, pnpm generate <pkg>, audit again, until the list is empty.

Two things the audit cannot judge:

• Typed patch configs. packages/aws/patches/<sdkId>.json is applyAwsSpecPatches config, not RFC-6902; the audit reports those files as not audited. Judge them by reading the model.
• Whether a needed patch is right. A patch that still changes the model may still be wrong about the wire. Patches that encode observed behaviour — error statuses the spec omits, required fields the API actually omits, x-sensitive marks — are only confirmable against the live API. With credentials, probe: create throwaway resources, hit the mutating routes with invalid bodies, record the statuses, tear down promptly. Add only what you observed; a probe that never reached validation (401/404 on a dummy slug) proves nothing. Writing the patch that records it is the distilled-sdk-patch skill.

While slimming, keep each file's description true to what is left, and fold files of the same kind together when they shrink to a few ops (two files that both only add omitted 422s are one file). The description is the bug report you will send upstream.

Step 4 — check the delta

git diff --stat -- packages/<pkg>
pnpm exec tsc -b packages/<pkg> --noCheck false   # the root typecheck:ci is heavy
pnpm lint

Read git diff -- packages/<pkg>/src/services for what upstream changed underneath you, beyond the patches:

• Renamed or removed operations. An upstream operationId change renames an export; a removed path removes one. Both are breaking and belong in the PR body by name, old → new.
• Nullability and required flips on shapes callers already use — T → T | null in an output is a type break too.
• Operation count before and after, as one line.

Step 5 — commit and PR

Stage explicit paths — the gitlink, .generated-specs/, patches/, src/services/, and README.md if an example changed:

git add -- packages/<pkg>/specs/spec-mirror-<pkg> packages/<pkg>/.generated-specs \
  packages/<pkg>/patches packages/<pkg>/src/services
git commit -m "feat(<pkg>): regenerate from the <YYYY-MM-DD> spec"

The PR body records what the next person needs to reproduce and review:

• mirror commit before → after, and the date of the spec
• operations before → after
• patches deleted, one line each with why (already in the spec, never read by convert, overwrote what upstream now publishes)
• patches kept, one line each with what the spec still gets wrong
• renamed / removed operations

That "patches kept" list, with each file's description and a link to it on the branch, is also the message to send the provider. Group it by kind — missing x-nullable, shared models with too many required fields, undocumented error statuses, a wrong response schema, secrets with no sensitive mark — because that is how they will fix it.