Pre Release Sweep

Pre-Release Sweep

Verify every change merged since the last release tag actually works in the browser, on a stack built from current origin/main, and leave a tracking doc a release cut can cite. The tracking doc is the deliverable; the browser work exists to fill it.

Tracking doc: docs/testing/release-sweeps/<YYYY-MM-DD>-<from-tag>-to-<to>.md (copy TEMPLATE.md in that directory). Write to it as you go — a sweep outlives a context window.

Phase 0 — Gate on in-flight work

1. gh pr list --state open — non-draft PRs against main with CI running are the "waiting" set. Draft/stacked PRs are not.
2. gh pr checks <n> --watch in the background (never poll by hand).
3. Merge only on head-SHA green (bare gh pr merge <N> enqueues; never --admin). Ignore only known-noise checks and say which: Trivy on CVE-2026-14456 (base-image openssl) is the current one — confirm the CVE id in the log before dismissing it.
4. Record the final origin/main SHA in the doc header. That SHA is the base of the sweep; anything merged after it is a follow-up entry, not silently absorbed.

Phase 1 — Stack at main (not the feature branch)

pnpm wt-stack up builds from the worktree's checkout. A sweep on a feature branch verifies the wrong code (bit us 2026-08-24, 47 commits stale).

git fetch --deepen=200 origin main            # herdr worktrees are shallow
git checkout -b qa/sweep-post-<from-tag> origin/main   # fixes land here
sed -i '' 's|^BREEZE_DOCKER_SUBNET=.*|BREEZE_DOCKER_SUBNET=<fresh /24>|; s|^BREEZE_CADDY_IP=.*|BREEZE_CADDY_IP=<same /24>.10|' .env
source ~/.nvm/nvm.sh && nvm use               # .nvmrc pin, not host node
pnpm install --frozen-lockfile                # code-mounted node_modules — stale tree = Astro NoMatchingRenderer
pnpm wt-stack up                              # read .breeze-stack.json for baseUrl + creds

Pick the /24 from docker network ls -q | xargs docker network inspect --format '{{.Name}} {{range .IPAM.Config}}{{.Subnet}}{{end}}' — an existing network with the same subnet fails compose with "Pool overlaps", running or not.

Boot with every feature flag the inventory (Phase 2) names, or the surface is invisible and the check reads as a false FAIL. If the API is unhealthy with a does not provide an export named … error, the dev image is stale — pnpm wt-stack up --rebuild. The seed has one org and a non-platform-admin; promote it for /admin/* rows (update users set is_platform_admin=true where email='admin@breeze.local') and let a sweep agent create sibling orgs. Then log in yourself once before dispatching any browser agent; a blocked sweep agent is 15 wasted minutes.

Stack traps seen 2026-09-08 (v0.110→main sweep): (1) the first wt-stack up on a fresh DB fails with "api is unhealthy" because the healthcheck window closes during the ~645-migration replay — wait for Applied N migration(s) in docker logs <api> then run up again, it is idempotent; (2) set WEBAUTHN_RP_ID=localhost in the sweep .env or every passkey ceremony throws SecurityError (rp.id comes back as the prod domain) — the CDP virtual authenticator does not help; (3) SMS has no local provider (POST /auth/phone/verify → 501), phone-MFA rows are BLOCKED by design, say so; (4) role-forced MFA (force_mfa on Partner Admin) can lock the seeded admin into /auth/mfa/setup?forced=1 — MFA_FORCE_FOR_PARTNER_ADMIN=false is the relief valve (default OFF since #5307); (5) the seed's Invite offers partner-scoped roles only, so an org/site-restricted reader must be created via SQL; (6) fixer first-passes regress: run one review round per fix even when the diff is small — two of nine fixes this sweep needed it.

Stack traps seen 2026-09-29 (v0.118.2→main sweep): (1) a fresh worktree .env copied from .env.example has FORCE_HTTPS=true, NODE_ENV=production and a placeholder PUBLIC_API_URL, and the dev override defaults PUBLIC_API_URL to the prod domain — every API POST returns 400 until you set FORCE_HTTPS=false, TRUST_PROXY_HEADERS=false, and PUBLIC_API_URL / PUBLIC_APP_URL / DASHBOARD_URL / CORS_ALLOWED_ORIGINS to the local base URL; (2) the dev override defaults BREEZE_WORKSPACE_ENABLED to true — set it false on a stack without pgvector; (3) when the Playwright MCP browser is held by another session, drive headless Playwright scripts instead: save storageState after every script (refresh tokens rotate, and a script that dies before saving leaves the next run logged out) and set localStorage['breeze-onboarding-complete']='true' to skip the onboarding overlay.

Phase 2 — Change inventory (sonnet, no browser)

Dispatch one Explore agent (model: sonnet) to produce the inventory table: merged PRs since the tag, surface (web / portal / api-only / agent-only / docs-chore), a concrete click-path read from source (real route from apps/web/src/pages, real button labels), the visible outcome that proves the change, and prereqs (flag, role, live agent) that make it BLOCKED locally. It must also list every BREEZE_*/*_ENABLED env var in the diffs and cluster the click-paths into 4–6 area groups.

Paste the table into the doc with a Status column = TODO. Every row ends the sweep as PASS | PARTIAL | FAIL | BLOCKED | N/A — never left TODO.