Pi Loop Forensics

Loop Forensics

Use when a run dies with agent loop aborted: ..., or a model burns a long turn thinking without ever calling a tool. This skill decides why before anything is changed: the abort message names a symptom, not a cause.

Sibling skill pi-check-session-logs covers tool call errors. This one covers repetition and runaway turns. They do not overlap.

Where the evidence lives

Log rows are {"time","type","content",...} with type in session_start, thinking, llm_text, tool_call, tool_result, user, error. The model name is on the session_start row — always record it, findings are per-model.

How the guard actually works

stuckDetector (internal/tui/agent_loop.go:119) has three independent arms:

• observe — identical consecutive tool calls, trips at maxRepeatToolCalls=10. Pagination args are stripped first (volatileToolArgs), so paging one file collapses to one fingerprint.
• observeError — same tool failing maxToolErrorStreak=10 times running.
• observeOutput (:246) — the model's own text/thinking. Needs a period ≥ minOutputPeriod=16 bytes, ≥ minPeriodVariety=8 distinct bytes, and maxOutputRepeats=12 byte-exact back-to-back copies inside an 8 KB tail.

The third arm is the only one that sees a turn making no tool calls at all. outBuf is never reset across a run, and the thinking branch (:690) skips dedup.SkipText, unlike the text branch (:697).

Timeline caveat — check this before judging any old log. Sessions predating these commits cannot be compared against current behavior:

Steps

1. Sweep the corpus for scale and per-model distribution:

   python3 .pi-go/skills/pi-loop-forensics/scan_logs.py

   Reports per model: session count, aborts, longest tool-free thinking run (max/p95), worst periodic repetition. Also lists aborted sessions and the top offenders. Loops are almost always concentrated in one model.

2. Score individual logs:

   python3 .pi-go/skills/pi-loop-forensics/score_run.py ~/.pi-go/log/*/session-*.log

   Three metrics per log: think_run (longest tool-free thinking run), reps/period (longest byte-exact periodic tail), intent/calls ("let me write/run/test" phrases vs actual tool calls). A high intent:calls ratio is the signature of announce-but-never-act.

   reps>=12 is the reliable discriminator. The think_run>=20 arm is heuristic and does produce false positives on models that legitimately think in long bursts — confirm with reps before calling it a loop.

3. Discriminate the cause. Run all three tests; do not stop at the first plausible one.

   • Race / double-emission? Hash every thinking payload in the session and count exact duplicates, and check timestamps. Genuine model output has zero duplicate payloads, varied lengths, and monotonic timestamps spaced by stream latency. Duplicated payloads or identical timestamps would mean pi-go re-emitted chunks — a real bug in the stream path.
   • Tool-parse failure? Grep thinking and text for tool-call syntax that leaked in as prose: <tool_call, <function, tool_calls, "name":...,"arguments", ```json, <think>, <｜tool. If the model emitted a call the provider layer failed to parse, the raw syntax shows up in a text channel and the model never receives a result — which looks exactly like a loop. Zero matches rules this out.
   • Guard too low? Compare the tripping value against the corpus. Healthy sessions peak around 1 periodic repeat; the threshold is 12 byte-exact copies. If healthy runs sit far below the threshold, the guard is not the problem.

   If all three are ruled out, it is inference-level repetition collapse, and the fix is a provider/sampling question, not a pi-go parsing question.

4. A/B replay across providers. Find a seed session whose last persisted event is the tool result immediately before the spiral — the degenerate thinking usually never gets committed to events.jsonl, so resuming restores the exact pre-failure state and the next turn is the one that broke.

   # From an isolated worktree — a resumed agent can write files.
   PI=/path/to/pi TRIALS=3 OUT=/tmp/ab-replay \
     bash .pi-go/skills/pi-loop-forensics/ab_replay.sh

   Arms are ollama-cloud, ollama-local, opencode (override with ARMS="ollama-cloud opencode"). Each arm runs a preflight one-shot first and is skipped with its error if credentials are dead, so a bad key costs one call instead of every trial. Override the seed with SEED=<session-id>.

   Per trial the script copies the seed to a throwaway session ID (the original is never mutated), rewrites meta.json (id/model/provider/workDir), resumes with --trace-http --mode print, scores the log, and deletes the copy.

   • Nothing pins temperature or seed, so reproduction is probabilistic — run several trials per arm and compare rates, not single outcomes.
   • --trace-http writes full request/response bodies to the session log. Credentials are masked (internal/provider/provider.go:440); prompts and source context are not.
   • Run from a git worktree, never the primary checkout: a resumed agent can write files.