Regression Hunt

Regression Hunt

Track down a behavior that used to work and now fails, changed, or regressed. Use this when a bug report points to a recent breakage, especially when the cause is not obvious yet.

Arguments

$ARGUMENTS: a short description of what regressed.

Helpful extras when available:

• failing test name or file
• error message or log line
• expected vs actual behavior
• suspect PR, branch, commit, or file
• reproduction command, input, route, or endpoint

A plain-English bug report is enough to start.

Instructions

1. Restate the regression clearly

• what used to work?
• what is broken now?
• what is expected instead?

2. Build a feedback loop (this is the skill)

Everything else is mechanical. With a fast, deterministic, agent-runnable pass/fail signal, bisection and hypothesis testing all just consume it. Without one, no amount of staring at code will save you. Spend disproportionate effort here. Be aggressive. Be creative. Refuse to give up.

Resolve the repository's test runner and the flags its scripts wire from its instructions first; the commands below show the Bun shape. Try these in roughly this order:

1. Run the failing test via the package's test script, e.g. bun run test -- --bail -t "<pattern>", so flags wired into the script (--preload, custom setup) are preserved; calling bun test directly bypasses them. Avoid bun --bun test from a worktree root. Note that bun test positional arguments are file path patterns, not test names; use -t "<pattern>" to filter by test name and --bail to fast-fail.
2. Curl / HTTP script against the backend dev server. Wrap fetches in AbortSignal.timeout(10_000) so a hung request does not rot the loop.
3. CLI invocation diffing stdout against a known-good snapshot.
4. Headless browser (Chrome DevTools MCP, Playwright, or similar) against the frontend dev server.
5. Replay a captured trace (saved payload, event log) through the code path in isolation.
6. Throwaway harness exercising just the bug code path with a single function call. For DB-touching code, a fresh test database per loop is usually faster and more honest than mocking.
7. Differential loop: same input through old-commit vs new-commit, diff outputs.
8. Bisection harness against the loop above. Drive it through the package script so wired flags survive, e.g.: git bisect run bun run test -- --bail -t "<test-name>". Verify the harness actually fails on a known-bad commit before starting: bun test exits 0 when no test names match, which would silently mark every commit as good and produce the wrong culprit.
9. Property / fuzz loop if the bug is "sometimes wrong output".

Then treat the loop as a product. Iterate on it:

• Faster? Cache setup, skip unrelated init, narrow scope.
• Sharper signal? Assert on the specific symptom, not "didn't crash".
• More deterministic? Pin time, seed RNG, isolate filesystem, freeze network.

A 30-second flaky loop is barely better than no loop. A 2-second deterministic loop is a debugging superpower.

For non-deterministic bugs the goal is a higher reproduction rate, not a clean repro. Loop the trigger 100×, parallelise, narrow timing windows, inject sleeps. A 50%-flake bug is debuggable; 1% is not. Keep raising the rate until it is.

If you genuinely cannot build a loop, stop and say so explicitly. List what you tried. Ask the user for: environment access, a captured artifact (HAR, log dump, screen recording with timestamps), or permission to add temporary instrumentation. Do not proceed to hypothesise without a loop.

3. Encode the regression in a test before fixing it

Only if a correct seam exists, one where the test exercises the real bug pattern as it occurs at the call site. A test at the wrong seam (too shallow, single-caller test when the bug needs multiple callers, unit test that cannot replicate the trigger chain) gives false confidence.

If no correct seam exists, that itself is the finding. Note it and flag the architectural gap for step 10. The codebase is preventing the bug from being locked down.

Prefer focused integration tests over deep unit tests when the regression crosses layers. If the area has no automated harness, build the smallest reproducible check and say why a proper regression test was not added yet.

4. Generate 3–5 ranked hypotheses before instrumenting

Single-hypothesis generation anchors on the first plausible idea. Each hypothesis must be falsifiable; state its prediction:

If <X> is the cause, then <changing Y> will make it disappear / <changing Z> will make it worse.

If you cannot state a prediction, the hypothesis is a vibe; sharpen or discard. Show the ranked list to the user before testing; domain knowledge often re-ranks instantly ("we just deployed a change to #3"). Don't block on it if the user is AFK; proceed with your own ranking.

5. Inspect recent change history

• current diff
• recent commits touching the area
• suspect PRs or refactors
• React Compiler: if the regression is a re-render / stale-closure / perf shift in the frontend, check whether the compiler's output for the affected component changed. A lot of "this used to memoize and now it doesn't" bugs live here, not in your hand-written code.

Use git history when helpful, but do not stop at blame; verify the actual cause. For regressions specifically, git bisect run against your step 2 loop is often the fastest path to the offending commit.