Retrospect External Babysitter Run

Retrospect External Babysitter Run

Analyse a babysitter run that lives in an external public repository, using the same lens as the in-repo /babysitter:retrospect command. Produce a written retrospective with concrete suggestions for the process author (or, if the insight generalizes, for the babysitter project itself).

When to use

• User names an external repo and asks for a retrospective.
• User asks "find a babysitter run to retrospect on" (combine with the catalog-babysitter-users skill to pick one).
• User asks "how are other people using babysitter processes? What do they get wrong?".

Prerequisites

• gh CLI authenticated.
• docs/repo-with-babysitter-processes.md exists (if not, run the catalog-babysitter-users skill first).
• A workspace directory where external repos can be shallow-cloned (default: /tmp/babysitter-retrospect/ or .a5c/tmp/external-runs/).

Phase 1 -- Target selection

1. Read docs/repo-with-babysitter-processes.md and list Active repos with stars + description. If the user already named a repo, skip to step 3.
2. Ask the user which repo to retrospect on (use AskUserQuestion in interactive mode; if non-interactive, pick the highest-starred Active repo that wasn't retrospected in the last 30 days -- track via docs/retrospectives/<owner>-<name>/log.md).
3. Confirm the target with the user before cloning.

Phase 2 -- Discover processes and runs

Shallow clone the target:

mkdir -p .a5c/tmp/external-runs
cd .a5c/tmp/external-runs
gh repo clone <owner>/<name> -- --depth 50 --single-branch
cd <name>

Locate:

• Process files: files importing defineTask from @a5c-ai/babysitter-sdk. Use Grep: grep -rl "from '@a5c-ai/babysitter-sdk'" -- . --include='*.js' --include='*.ts'.
• Committed runs: .a5c/runs/<runId>/ directories. Many repos gitignore .a5c/runs/ entirely -- that's fine; note it and proceed with process-only retrospective. When runs ARE committed, look for run.json, journal/, tasks/, state/output.json.
• Historical runs via git log: git log --all --diff-filter=A --name-only -- '.a5c/runs/' surfaces runs that existed at some commit even if later cleaned up. Check out the commit that introduced the run if you want the journal content.

Summarize to the user:

• N process files found, by top-level directory
• M run directories present in HEAD; P additional historical runs reachable via git history
• Which runs completed vs failed (grep RUN_COMPLETED / RUN_FAILED in the journal)

Phase 3 -- Pick a run to retrospect

If multiple runs exist, ask the user (interactive) or default (non-interactive) to:

• The most recent failed run (highest signal for process improvement), OR
• If no failures, the most recent completed run.

If no runs are committed at all, switch to a process-only retrospective: analyse the process file(s) for quality issues without run evidence. Mark the output clearly as process-only.

Phase 4 -- Load the run

Inspect, in order:

• .a5c/runs/<runId>/run.json -- processId, entrypoint, prompt, createdAt
• .a5c/runs/<runId>/inputs.json -- user intent
• .a5c/runs/<runId>/journal/*.json -- event stream (RUN_CREATED, EFFECT_REQUESTED, EFFECT_RESOLVED, RUN_COMPLETED / RUN_FAILED). Read every journal entry; it is the authoritative record.
• .a5c/runs/<runId>/tasks/<effectId>/task.json + result.json -- per-effect definition and result
• .a5c/runs/<runId>/state/output.json (if present) -- final output
• The process file referenced by run.json.entrypoint -- cross-reference against the journal to see what the author intended vs what happened.

Phase 5 -- Retrospective analysis

Mirror the in-repo /babysitter:retrospect workflow. Produce notes under each heading:

5.1 Outcome

• Success / partial success / failure.
• Total iterations, duration, distinct effect count, retry count.
• Final output quality (from state/output.json shape + content).

5.2 What went well

• Effects that resolved on first try.
• Process sections with clear inputs/outputs and no re-runs.
• Useful breakpoints that caught real issues before they propagated.

5.3 What went poorly

• Effects that were re-dispatched (same invocationKey or similar taskId appearing repeatedly).
• Long gaps between EFFECT_REQUESTED and EFFECT_RESOLVED (external bottlenecks).
• Breakpoints that looped (approval -> reject -> retry -> reject).
• Tasks that crashed and what the error category was (Configuration / Validation / Runtime / External / Internal).
• Any RUN_FAILED: trace the last few events and the thrown error.