Debugging

Debugging

Feedback Loop FIRST: No hypothesis without a repro loop. No fix without root cause. No fix without blast radius scan.

Reference Files

• references/investigation-hygiene.md — investigation discipline, evidence handling
• references/root-cause-playbooks.md — scenario-specific debugging playbooks

Feedback Loop FIRST (Before Any Hypothesis)

A hypothesis without a repro loop is a guess. Before H1, build a fast, deterministic, agent-runnable signal that turns red on the bug.

Construction Ladder (try in rank order, stop at first that works — ordered by loop tightness: earlier rungs are faster and more deterministic)

1. Failing automated test (unit/integration) — best: lives at a seam, reusable as RED
2. curl/HTTP request with asserted response
3. CLI snapshot diff (run command, diff stdout/stderr/exit)
4. Headless browser script (real DOM/runtime crash)
5. Trace replay (recorded request/log/event re-run)
6. Throwaway harness (tiny script calling the suspect function)
7. Property/fuzz check (when failing input is unknown)
8. git bisect run (regression with existing test)
9. Differential old-vs-new (last-good vs HEAD behavior diff)
10. Human-in-the-loop (LAST resort: scripted manual steps)

Tighten the loop — treat it as a product. Once you have a loop, keep tightening:

• Faster? Cache setup, skip unrelated init, narrow the test scope — sub-second beats sub-minute.
• Sharper signal? Assert the exact failing fact, not a noisy superset — never just "didn't crash".
• More deterministic? Pin time, seed RNG, isolate filesystem, freeze network — same input → same red, no drift. A 30-second flaky loop is barely better than none; a 2-second deterministic one is a debugging superpower.

Red-capable completion criteria — the loop is done when you can name one command (a script path, a test invocation, a curl) that you have already run at least once (paste the invocation and its output), and it is:

• Red-capable — drives the actual bug code path and asserts the user's exact symptom (can go red on this bug, green once fixed). Not "runs without erroring".
• Deterministic — same verdict every run (flaky bugs: a pinned, high reproduction rate).
• Fast — seconds, not minutes.
• Agent-runnable — you can run it unattended. No red-capable command, no hypothesis phase. If you catch yourself reading code to build a theory before this command exists, STOP.

Flaky bugs: run in a tight loop (for i in $(seq 1 N); do ...; done), record hit rate (e.g. 3/50), treat raising that rate as loop iteration.

When You Genuinely Cannot Build a Loop

STOP. Do NOT advance to hypothesis. Return BLOCKED with:

• What was tried: each rung attempted and why it failed
• Concrete ask: the one thing that would unblock (env/credential access, captured artifact, permission for temporary instrumentation)

LSP-Powered Root Cause Tracing

Use LSP to trace root causes through the codebase:

• Go to Definition — follow the call chain to where the value is actually set
• Find References — find all callers of a suspect function (blast radius)
• Go to Type Definition — check if the type allows the failing value
• Hover — check types and signatures at the failure site

Don't guess where a value comes from — trace it with LSP. Don't grep for a function name — use Find References to get every caller with type info.

The Four Phases

Phase 1: Root Cause Investigation

1. Understand — expected vs actual, when did it start?
2. Git History — git log --oneline -20 -- <files>, git blame, git diff BASE..HEAD
3. Compounded knowledge — if docs/solutions/debugging/ exists, check for a prior write-up matching this symptom before starting fresh investigation
4. LOG FIRST — collect error logs, stack traces, run failing commands. The error text is the highest-density evidence you will ever get; acting first destroys or masks it.
5. Before-capture — persist step 4's capture while the bug is reproducing: save the red output (log excerpt, command output, screenshot) to a durable artifact BEFORE writing the fix. It is the "before" half of the evidence pair; the fixed state alone proves less, and the failing state is cheapest to capture while it is red.
6. Feedback Loop — build repro signal (construction ladder above). No loop → fail closed.
7. Variant Scan — identify which variant dimensions must keep working (locale, config, env, platform, data shape, concurrency). A fix verified on one variant routinely breaks a sibling variant.

Repro Minimisation: After reproducing the bug, shrink to the smallest scenario that still goes red before forming hypotheses. Cut inputs, callers, config, and environment one at a time. Re-run after each cut. Every remaining element is load-bearing — removing it should make the bug disappear.

Why: A minimal repro shrinks the hypothesis space. The fewer moving parts, the fewer places the bug could hide.

Phase 2: Pattern Analysis

1. Read the code around the failure — not just the failing line, the surrounding logic
2. Check for recent changes — git diff the files involved
3. Look for similar patterns — grep for the same anti-pattern elsewhere
4. Identify the mechanism — not "what's wrong" but "how does the wrong thing happen"

Phase 3: Hypothesis and Testing

Generate 3-5 ranked hypotheses (H1, H2, ...) with 0-100 confidence BEFORE testing any of them — fewer than 3 means you anchored. Rank by explanatory power — which hypothesis explains the most symptoms with the fewest assumptions. Testing the first plausible hypothesis anchors you; generating multiple first prevents anchoring bias and surfaces connections between hypotheses. Proceed to fix only when one reaches 80+.

Hypothesis Quality Criteria:

• States a specific mechanism ("X returns null because Y is not set when Z")
• Predicts a specific test outcome ("if I set Y, X returns the correct value")
• Is falsifiable ("if Y is already set, this hypothesis is wrong")
• Explains ALL observed symptoms, not just the primary one

Instrumentation preference: Each probe must map to a specific prediction. Change one variable at a time. Tool preference:

1. Debugger / REPL inspection if the env supports it — one breakpoint beats ten logs.
2. Targeted logs at the boundaries that distinguish hypotheses.
3. Never "log everything and grep". Tag every debug log with a unique prefix (e.g. [DEBUG-a4f2]) so cleanup is a single grep.

Performance branch: For performance regressions, logs are usually wrong. Establish a baseline measurement first (timing harness, performance.now, profiler, query plan), then bisect. Measure first, fix second.

Hypothesis Confidence Scoring:

When to Restart Investigation: If 3 hypotheses fail, you're pattern-matching, not investigating. Re-read the loop output. Re-trace with LSP. Consider you're looking at the wrong layer. If two or more fixes that share one premise failed the same gate, write the premise down — the premise, not the next fix variant, is now the suspect. Ask what the premise predicts beyond that gate: where the failure should concentrate, which variant should stay green. Test the prediction with the repro loop before the next fix; a prediction the observation contradicts disposes of the premise, not just the fix.