Aicr Reviewing Component Drift

Step 3 — Gather evidence per component

Ordered by how often each is what actually bites.

1. Values-path compatibility. Pull both chart versions and diff their values, then verify every path AICR depends on still exists:

   • keys set in recipes/components/<name>/values.yaml
   • values: blocks in recipes/overlays/*.yaml and recipes/mixins/*.yaml that target the component
   • the component's nodeScheduling.nodeSelectorPaths and tolerationPaths in recipes/registry.yaml

   The third is the silent one: a renamed master.nodeSelector leaves AICR rendering valid YAML that no longer schedules anything correctly, and no current test catches it. Treat a missing path as a blocking finding.

2. Lockstep families. These move together; never recommend a partial bump:

   • slinky-slurm-operator-crds, slinky-slurm-operator, slinky-slurm
   • mariadb-operator-crds, mariadb-operator, slurm-accounting-mariadb
   • agentgateway-crds, agentgateway
   • prometheus-operator-crds, kube-prometheus-stack, keyed on appVersion, not chart version (the two chart sequences are unrelated). A lagging CRDs pin also breaks nvsentinel and k8s-ephemeral-storage-metrics, which create resources defined by those CRDs. TestPrometheusOperatorAppVersionLockstep enforces this.
   • prometheus-adapter / prometheus-adapter-ocp, nvidia-dra-driver-gpu / nvidia-dra-driver-gpu-ocp, and k8s-nim-operator / k8s-nim-operator-ocp — these three -ocp twins carry a defaultRepository identical to their base, so BuildReport already collapses each pair into one report row; name both in the verdict anyway. The other four -ocp components (gpu-operator-ocp, network-operator-ocp, nfd-ocp, cert-manager-ocp) have defaultRepository: "" and are manifest-only — BuildReport drops them before Tracked++, so they never appear in the report at all (they surface in untrackedComponents instead). Do not assume a base component's row already accounts for one of these four; there is no shared row to collapse into.

3. Image and BOM delta. Render old against new and diff the image list. make bom-docs is the repo's renderer; a new image means new vulnerability surface, and a new registry host means tools/registry-inventory's allowlist needs extending too.

4. CRD and upgrade-record impact. For CRD-bearing charts (kueue, gatekeeper, mariadb, slinky, prometheus-operator-crds, nvsentinel), check whether the transition needs an ADR-021 record: see pkg/upgrade and tools/check-upgrade-records.

5. Upstream breaking changes. Where the chart maps to a GitHub project, read the releases between the two versions (gh release list -R <owner>/<repo>), looking for removed flags, renamed values, and required migration steps.

6. Kubernetes compatibility and blast radius. Compare the new chart's kubeVersion against the K8s.server.version constraints of the overlays that pull the component, and check whether recipes/checks/<name>/health-check.yaml exists and whether its assertions still hold against the new chart's resource names.

Step 4 — Write the recommendation

Write Markdown to a temp file ("$TMPDIR"/aicr-drift-review-<date>.md) and summarize it in chat. Rank components by (blocking findings, then update type, then age of the pin). Per component:

• Verdict: take / hold / defer, one line of why. When the row carries alternatives, name which version the verdict is about — a bare "take" is ambiguous once more than one is on offer
• Evidence: the findings from Step 3 that produced the verdict, with the specific values path, image, or release note that matters
• Cost: what a bump would require — a values-file edit, a new upgrade record, a BOM refresh, a health-check update, an allowlist entry

Do not edit the registry, file issues, or open PRs. Ask before doing anything outward-facing.

Step 5 — Update the state file

Record every component reviewed: the version pair, the full candidate set the verdict was formed against, which candidate the verdict recommends, the date, and for a hold or defer the condition that would change it. Next week's digest repeats the same components by design — this file is what keeps the review from repeating with it.

candidates is what Step 1 compares against, so record it verbatim from the report rather than summarizing: an entry that omits a candidate makes next week's run confirm a verdict that never considered it. recommends is only meaningful when the row had alternatives; omit it otherwise, since the single candidate is already named by to.

Gotchas

• crane ls on ghcr.io works in the sandbox; NGC (nvcr.io, and helm.ngc.nvidia.com over curl) and some TLS paths need dangerouslyDisableSandbox: true. Try sandboxed first.
• OCI chart repositories list cosign signature and attestation tags alongside real chart tags. Ignore anything that is not a version.
• recipes/overlays/aks.yaml pins kube-prometheus-stack to 83.7.0 deliberately (#700, declared in versionPinExemptions in pkg/recipe/version_pin_guard_test.go). A kube-prometheus-stack bump has to reckon with that exemption; do not propose removing it casually.
• aws-efa's chart is tracked but its device-plugin image is in Renovate's ignoreDeps and must be coordinated with EKS add-on releases.