QA Session

QA Session — browser verification of a change

Turn a diff into UI steps, reproduce the old bug on main, then prove the fix removes it. Every step is derived from the code (routes, labels, gates) — never guessed.

What QA has to disprove is the reported symptom, not the diff. A fix can be correct, its tests green, its own control run red-then-green, and still leave the ticket's bug in place — the diff answers the analysis, and the analysis can have found a real but different defect. Everything below is built so that failure mode cannot end in a PASS.

Step 0 — Args (both required)

1. Mode: manual (operator clicks, you give steps + triage) or auto (you drive the browser).
2. Target: ISSUE-ID, PR number, branch/worktree, or free-text description of the surface to test.

Either missing → AskUserQuestion, stop until answered.

auto mode drives two browsers and they are not interchangeable. Never the built-in Browser pane, and never type credentials in either.

• Claude in Chrome (mcp__claude-in-chrome__*) carries the operator's session, so it reaches the app with no login. Use it for navigation, DOM and computed-style probes, and plain clicks. It has NO middle-click action at all, and its input channel can die mid-run, delivering zero events to the page while every call still returns success. Prefer browser_batch; re-read refs after every navigation (stale refs click the backdrop and close drawers).
• Playwright (mcp__plugin_playwright_playwright__*) has real CDP input: browser_click takes button: "middle" and modifiers: ["ControlOrMeta"], browser_press_key sends genuine keypresses, browser_tabs lists what a gesture opened. Its browser is a separate profile with NO app session, so ask the operator to log in there once — after that the whole real-gesture round runs unattended.

Any check whose verdict depends on a browser default — middle-click, modifier-click, a keypress — MUST run on Playwright. A dispatched MouseEvent or KeyboardEvent exercises the app's handlers and nothing else, so a round driven that way has verified the handler, not the gesture: report it as such.

Prove the instrument before trusting a negative. Install a capture listener, fire one harmless click, and confirm an event actually reached the page. A dead input channel reads exactly like an app ignoring the click.

Step 1 — Resolve the app under test

Resolve the target down to a worktree name first — that name is what the slot registry is keyed on and what the container names derive from:

• ISSUE-ID → $LOOP_STATE_DIR/<ISSUE-ID>/state.md (default ~/.claude/loop-state) for worktree, branch, port.
• PR number or URL → its head branch, then the state dir whose branch: matches, same as loop-revise:

  BRANCH=$(gh pr view <PR> --json headRefName --jq .headRefName)
  grep -l "^branch: ${BRANCH}$" "${LOOP_STATE_DIR:-$HOME/.claude/loop-state}"/*/state.md

• Branch, worktree name or free-text → the slot registry at the monorepo root, i.e. ../.worktree-slots from the front/ checkout (name:front_port:front_base:api_port:api_base), else docker ps.

lago-worktree keys a slot by the branch with / replaced by -, so normalize before looking one up — WT=${BRANCH//\//-} — otherwise every slashed branch misses its own slot and reads as "not running".

• Front URL http://localhost:<front_port>; containers lago_front_wt_${SAN} and lago_api_wt_${SAN}, with SAN=$(echo "$WT" | tr '[:upper:]' '[:lower:]' | sed 's/[^a-z0-9]/_/g'). An empty api_port in the slot means the worktree has no API of its own and proxies the shared stack, whose API is lago_api_dev.
• Stale code / 504 Outdated Optimize Dep / blank page → clear vite cache THEN restart (first load is slow):

  docker exec <container> sh -c 'rm -rf /app/node_modules/.vite' || true
  docker restart <container>

Step 2 — Preconditions

Read the ticket's own repro steps first, then the diff + spec — in that order, so the report frames the diff and not the other way round. Enumerate the ticket's attachments (get_issue → attachments, plus list_comments): a video, screenshot or Slack thread often carries the only repro there is. You cannot watch a video — say so explicitly, and ask the operator for the gesture and the surface rather than substituting a plausible one.

No ticket (PR, branch or free-text target): the report is whatever the operator handed over — PR body, their own message. Ask for the exact gesture, payload and surface when it is not there; never substitute a plausible one. Everywhere below, "ticket" and "spec" mean that source instead, and with no acceptance criteria the AC control (Step 4a) collapses into the report control — say so rather than deriving ACs from the diff.

Then check what gates the surface:

• Feature flag — featureFlag: FeatureFlagEnum.X on the route in src/core/router/*; enum value = DB string. Flip it on the organization whose slug is in the URL under test, in the API container resolved in Step 1 (lago_api_wt_${SAN}, or lago_api_dev on the shared stack). Organization.first is an arbitrary row: it can enable the flag on an org nobody is testing and leave the tested route gated.

  docker exec -it <api_container> bin/rails runner 'o = Organization.find_by!(slug: "<slug>"); o.update!(feature_flags: (o.feature_flags | ["<flag>"])); puts o.feature_flags.inspect'

• Permissions / premium — permissions: on the route, premiumIntegrations gates.
• Data — what must exist (customer, plan, wallet) and the org slug (URLs are /<slug>/...).

Step 3 — Test plan (UI language only)

Numbered steps, each with: exact URL · exact visible label (button, menu item, drawer title, read from the component and translations/base.json) · expected result · old-bug behavior. No internals in the steps.

• Check 1 is always the reporter's own path — their steps, their payload, their surface, described in their words. Diff-derived checks come after it, and are secondary.
• If the spec's acceptance criteria don't cover the reporter's path, that is already a finding: say so in the plan, and treat the report as the thing to satisfy. An AC set that only describes the diff means the analysis may have scoped a different defect.
• Target the exact code path the diff touched. Same feature ≠ same surface (in-editor preview toggle vs saved-version preview rebuild). Name which surface proves the fix.
• Test on the product surface, not a dev harness. /design-system/* pages differ structurally from the real one — they grow instead of scrolling, have no fixed-height/overflow-auto wrapper, no aside, no real data. Those differences are exactly what hides layout, scroll and clipping defects. Using a harness for speed is fine; list the structural deltas, then re-run check 1 on the real surface.
• Input fixtures: when the bug depends on content (paste, import, file), give the exact content, crafted against the diff — a generic fixture can silently fail to reproduce.
• Declared deltas: pull intentional visual changes and known follow-ups from the spec/PR in as "expected — not a bug" lines, so the operator doesn't file them.

Step 4 — Control run on main (mandatory, do it FIRST)

Reproduce the bug on unfixed code before verifying the fix — otherwise a PASS proves nothing. Two separate controls, both on main:

• (a) AC control — the spec's failing check. Proves the fix does what it claims.
• (b) Report control — the reporter's own steps, payload and surface. Proves the fix addresses what was actually filed.

They are not interchangeable, and (a) passing says nothing about (b). If (b) does not reproduce on main — the reported gesture works fine on unfixed code — then the analysis found a real defect that is not the reported one: stop, say it plainly, and hunt the reported symptom before signing anything off. Verdict is capped at PARTIAL until (b) reproduces.

Same trick both times:

1. Preferred (same worktree, git status --short must be empty) — revert the whole branch diff, which git checkout main -- <paths> cannot do for paths the branch adds, deletes or renames:

   git diff --binary main...HEAD > /tmp/qa-control.patch   # --binary: without it an image or font in the diff won't re-apply
   git apply -R /tmp/qa-control.patch   # worktree now runs main's code, vite HMR reloads
   # run the failing check, confirm the old behavior
   git apply /tmp/qa-control.patch      # back to the branch code — mandatory, see below

   Re-applying the same patch forward is what restores adds, deletes and renames symmetrically.

   The forward re-apply is not optional and never skipped. Every way out of the control run goes through it: the check reproducing, not reproducing, erroring, the browser leg dying mid-run in auto mode, and the stop above when (b) doesn't reproduce. Skip it and the branch is left running main's code under a dirty tree that reads as the operator's own uncommitted edits. Restore, confirm git status --short is empty, and only then report — including when reporting a stop.

2. Alternative: another running worktree whose branch doesn't touch those files (git diff main...HEAD --stat) — they share the DB, so the same fixture URL works on both ports.

Record which route was used. If neither is possible, say so plainly and do not claim causality.