Verify Smithy Service Parity

Verify Smithy Service Parity

Overview

Confirms a C2J-to-Smithy migrated service builds, packages, and stays API-compatible with the shipping SDK. Every step produces pasted evidence; the verdict is derived from that evidence, not from judgement.

The regenerated service files under sdk/ MUST NOT be committed in the migration PR — the release pipeline regenerates them. The migration itself is carried by the generator/ServiceModels/_smithy-migrated-services.json change and the generator/.DevConfigs/ changelog entry (plus any generator fixes the batch required). Staging them locally (git add) is still the right way to review: rename detection pairs each deleted C2J X.cs with its added Smithy X.g.cs, so git diff --cached -M -- <service trees> shows per-file rewrites instead of unrelated delete/add pairs. Stage one service at a time and unstage it (git restore --staged) once its verdict is recorded, so the index only ever holds the service under review.

Parameters

• service_name (required): SDK ServiceFolderName, e.g. Uxc, SSOOIDC, MarketplaceReporting.
• sdk_repo (required): path to the SDK checkout.
• comparer_repo (required): path to the build-utilities checkout containing AssemblyComparer.
• nuget_download_folder (optional, default: a new temp directory): scratch location for packages AssemblyComparer downloads.

Constraints for parameter acquisition:

• You MUST ask for all required parameters upfront in a single prompt.
• You MUST confirm service_name is present in <sdk_repo>/generator/ServiceModels/_smithy-migrated-services.json before starting, because the generator only emits Smithy output for services listed there.
• You MUST NOT hardcode any path not supplied as a parameter, because the SOP has to run on any machine and any checkout layout.

Steps

1. Regenerate

dotnet run --project <sdk_repo>/generator/SmithyDotNet/SmithyDotNet.Generator/SmithyDotNet.Generator.csproj -- --repo-root "<sdk_repo>"

Constraints:

• You MUST paste git status --short afterwards.
• You MUST confirm only <service_name> shows generated-file changes, because every migrated service regenerates on every run and changes to another service indicate an unintended generator change.

2. Compare the public surface against the C2J output

The C2J-generated code is the source of truth, because it is what ships today. Its version of every generated file is in git HEAD until this regeneration overwrote it, so retrieve each with git show HEAD:<path> and compare against the new output.

Constraints:

• You MUST list every public type, member and property from the C2J files and confirm each still exists in the Smithy output with the same name and the same .NET type, because a member that changes type or disappears breaks callers.
• You MUST compare signatures only, normalising away a constructor's : base(...) initialiser, an expression body (=> x versus { get { return x; } }), and whitespace, because none of those change the compiled public surface and treating them as findings buries the real ones.
• You MUST treat a property whose type narrowed or widened as FAIL, for example List<string> becoming string, since existing code stops compiling.
• You MUST compare the full [AWSProperty] and [AWS*] named-argument sets side by side for every member, because AssemblyComparer does not compare attribute argument values and step 7 will not catch a missing one — with one exception: Min/Max on a List<T>-typed member is inert on both sides (PropertyValueRulesWriter only emits rules for scalar members, and the consuming analyzer only validates compile-time-constant string/int literals), so a difference there is not a finding. A second exception: Min=0 dropped from a string member whose smithy.json @length trait declares only max — the C2J translation materializes the omitted min as 0, the Smithy trait is authoritative, and the analyzer's min check (value.Length < min) can never fire for a min of 0 — an empty-string warning requires min >= 1, and those members keep their Min (verify the trait in smithy.json before accepting). Every other attribute, including [AWSPaginator], has no such exception — its arguments are consumed outside this repo (AWS Tools for PowerShell reflects on it), so treat any drift there as a FAIL.
• To verify model traits, use the Smithy CLI instead of hand-parsing smithy.json. --allow-unknown-traits is required because the models reference aws.api# traits without shipping their definitions. Examples:
  • Length trait values for every string shape: smithy select --allow-unknown-traits --selector "string [trait|smithy.api#length]" --show-traits length <models>/<dir>/smithy.json
  • Members bound to response headers: smithy select --allow-unknown-traits --selector "member [trait|smithy.api#httpHeader]" <models>/<dir>/smithy.json
• You MUST confirm every C2J public type has a Smithy counterpart, including exception types, enumerations, paginators and the client interface, since a missing type is a removed API.
• You MUST NOT accept "the generator does not support this shape" as a reason for any difference in the public surface, because the migration is only allowed to change how the code is produced, not what it exposes.
• A public type or member present only in the Smithy output is not a finding when it exists in smithy.json and not in the C2J .api.json (verify both); give it a changelog line. Every other rule in this step still applies.
• For every marshaller and unmarshaller you MUST extract the set of names referenced in the C2J body — publicRequest.X, response.X, context.X, IsSetX(), and the wire names in Parameters.Add("x" / Headers.Add("x" — and confirm every one appears in the Smithy body. A name present in C2J and absent in Smithy is FAIL.
• Presence is not enough: you MUST also diff the literal wire strings character for character — the ResourcePath template, each AddPathResource("{token}", …) token, and every Parameters.Add("name" / Headers.Add("name" name. A changed literal is FAIL unless you prove it inert. The one inert case is a URI path-label token: a Smithy path label is keyed by the member name, so its {token} may differ from the historical wire label — a case/spelling change is acceptable ONLY when the AddPathResource key and the ResourcePath token change together and stay identical to each other, so the substituted path is byte-for-byte unchanged. A header or query name is wire-visible and has no such exception; any change there is FAIL.
• Min/Max/Pattern values and the ResourcePath template come from the Smithy model, which is the source of truth. Where the C2J translation differs from it (a Min=0 present on one side only, anchors on a pattern, a trailing slash on the path), the difference is EXPECTED.
• One accepted response-side difference: an exception member bound to an HTTP response header (@httpHeader on an @error shape; location: header in the C2J model). C2J emits a dead body read — context.TestExpression("<Header-Name>", targetDepth, ...) against a JSON key the service never sends — so the property is always null in the shipping SDK. The Smithy generator reads the actual header (context.ResponseData.IsHeaderPresent(...) / GetHeaderValue(...)), so the property starts being populated. This is acceptable ONLY when both models bind the member to the same header name (verify both), and it is a customer-visible behavioral fix: the migration's DevConfig entry MUST carry an additional changelog message stating the property is now populated. Decided 2026-09-11 for VPCLattice, ObservabilityAdmin and DSQL (RetryAfterSeconds from Retry-After; AmznErrorType from x-amzn-ErrorType).
• A matching name-set is not sufficient: for every reference in that set you MUST also compare the condition under which it executes — the enclosing if/guard, ternary, or unconditional placement — between the C2J and Smithy bodies line by line. A name present in both bodies but reachable under a different condition (a header added unconditionally where C2J guarded it, an IsSetX() check inverted, a branch that now fires for a different set of inputs) is FAIL, even though nothing is missing.
• You MUST run that body comparison even when every public signature matches, because a member dropped from marshalling changes no signature: the property still exists, callers still compile, and the value silently never reaches the service. No other step in this document detects it.
• You MUST verify any URL emitted in a doc comment resolves (or matches C2J's value exactly) — a broken <seealso href> is invisible to every other check in this document.