Anomaly Investigation

Anomaly Investigation Loop

A form → test → eliminate → confirm loop — root-cause analysis as a search. The artifact is an investigation log; the feedback signal is the count of live candidate explanations, driven down toward a single cause that is confirmed, not merely consistent. Each iteration you test one candidate against the data and drop the ones the data refutes, narrowing the field until one survives.

The discipline this enforces: a cause is "root" only when it both survives an honest attempt to refute it and makes a positive prediction that checks out (e.g. "if this is the cause, removing it restores normal" — and it does). A story that merely could explain the anomaly is a hypothesis, not a finding.

When to use

Use this when an anomaly is already in hand — you know roughly what looks wrong and want the cause diagnosed by elimination against the data. Default to a broad initial slate of mutually distinguishable causes, then test the one that splits the field fastest; if the anomaly is vague, your first job is to make it precise (iteration 0). Not for open-ended exploration of a dataset with no anomaly to chase (use data-analysis), and not for verifying an external claim against the literature (use claim-verify).

Setup

Resolve bindings interactively. If loop.run.yaml exists in the working dir, load it, confirm the values in one line, and skip to the loop. Otherwise: on Claude Code (the AskUserQuestion tool is available) infer a likely value for each binding and present it as the recommended option; on other hosts ask each as a quoted plain-text prompt. Then write loop.run.yaml (format: examples/run.example.yaml) and confirm the values before creating any other files.

Analysis snippets run in the user's environment via <analysis_cmd>, so they may use whatever the user has installed. Keep helper code stdlib-first (csv, statistics): if a snippet needs pandas/numpy, probe with try/except ImportError and degrade to a stdlib path, or offer a consented uv pip install "pandas==<ver>" — never assume the package is installed.

The loop

Copy this checklist and tick items off:

• Iteration 0 — characterize the anomaly precisely; form an initial slate of candidate causes in <log>.
• Pick a candidate to test (the one whose test most cleanly splits the remaining field).
• Test it: write <sandbox_root>/iter<N>/test.py, run with <analysis_cmd>, redirect to out.txt.
• Eliminate (data refutes it → drop from the live set) or advance (data supports it → keep it live).
• Confirm the survivor: when one candidate leads, run a positive test only it predicts, after a refutation attempt.
• Append a ledger row; stop when one cause is confirmed or at <budget>.

Iteration 0 — characterize. Quantify the anomaly precisely: write and run a snippet that pins down what deviated, where/when, and how big the deviation is against the normal baseline (the same metric on surrounding periods/segments). Then form an initial slate of candidate causes — mutually distinguishable explanations, broad enough to contain the truth (a real change, a composition/mix shift, a data-quality bug, a measurement change, seasonality, an outlier segment). List them in <log> as the live candidates. Record nothing as confirmed yet.

Then, until stop (one confirmed cause, or budget):

1. Pick a candidate to test. Ideally the one whose test most cleanly splits the remaining field, so each iteration removes as many live candidates as possible.
2. Test it. Write <sandbox_root>/iter<N>/test.py that computes the thing that would refute or support it (slice by segment/source/time, recompute the metric, compare distributions). Run it with <analysis_cmd>, redirecting output to <sandbox_root>/iter<N>/out.txt (never flood your context).
3. Eliminate or advance.
   • Data refutes it → mark it eliminated in <log> with the evidence; drop it from the live set.
   • Data supports it → keep it live, and if it is now the leading candidate run a confirming test: a positive prediction it uniquely makes (e.g. "remove / seasonally-adjust the suspected factor → the anomaly disappears"). Also try to refute it — a leading candidate that survives a genuine refutation attempt and passes its confirming test is the root cause.
4. Log one ledger row and continue, narrowing the live set.