Analyzing Claude Code Sessions

Analyzing your Claude Code sessions

Claude Code writes a full JSONL transcript of every session to ~/.claude/projects/. That is a complete record of what an agent was asked to do, every tool call it made, what failed, and what it cost. This skill turns that exhaust into an evidence-backed report.

The core pipeline is deterministic Python (gaia.factory.harvest) — no LLM, no network. Two optional steps do call a model through the local Claude Code CLI: classify (use-case labels) and synthesize (the written analysis). Everything derived is written to ~/.gaia/cache/factory/ and must never be committed — transcripts contain absolute paths, branch names, and whatever the user pasted into a prompt.

Run it

Redirect into the cache directory, never into a repository working tree. These tables are built from real transcripts, and an untracked tables.md sitting at a repo root is one git add -A away from being published.

FACTORY=~/.gaia/cache/factory

# 1. Extract. Deterministic, no LLM, no network. Roughly linear in transcript count.
python -m gaia.factory.harvest.scan

# 2. Tables. Absolute counts and share-of-total for every figure.
python -m gaia.factory.harvest.report > "$FACTORY/tables.md"

# 3. With use-case labels (see "Classifying intent" below):
python -m gaia.factory.harvest.report --labels "$FACTORY/labels.txt" > "$FACTORY/tables.md"

# 4. Per-request prompt size and local KV-cache memory. Re-reads the raw
#    transcripts, because steps 1-2 aggregate per session and that hides how
#    large any single request got.
python -m gaia.factory.harvest.context --labels "$FACTORY/labels.txt" > "$FACTORY/context.md"

# 5. What each proposed fix would actually save, in tokens and dollars.
python -m gaia.factory.harvest.savings > "$FACTORY/savings.md"

# 6. Optional, needs the Claude Code CLI: label use cases, then write the analysis.
python -m gaia.factory.harvest.classify
python -m gaia.factory.harvest.synthesize > "$FACTORY/analysis.md"

Only scan takes --root (transcripts elsewhere); scan and classify write with --out. Every other step reads the cache with --cache, and context and savings also take --projects if the raw transcripts are not under ~/.claude/projects.

Outputs in ~/.gaia/cache/factory/:

The five things that will mislead you

Learned by getting each one wrong first.

1. Subagents are not sessions. A delegated Task/Agent run gets its own transcript under <session-uuid>/subagents/. A plain */*.jsonl glob misses them entirely, and they can carry a large share of all tool calls in a delegation-heavy corpus. iter_traces attaches them to the parent; use Trace.walk() to include them and say explicitly which scope a number covers.

2. Identity must hash the full arguments. Keying a tool call on one argument (the file path, say) makes consecutive different edits to one file look identical. That single mistake overstated a "thrash" metric ~40×. Step.arg_hash covers the whole argument object; arg_digest is for display only. Never compute identity from the digest.

3. Order your error taxonomy specific-before-generic. Matching is first-wins, and a timeout also carries a non-zero exit code. With command_failed above timeout, real timeouts — often the largest failure class — get filed as generic command failures.

4. Rank savings by carried tokens, not by call count. A prompt is the whole conversation so far, so a result emitted at step 5 of a 50-step run is re-sent 45 more times. That carry multiplier decides what a fix is worth. Expect the two rankings to disagree: re-reads can be a large share of reads by count yet a small share of tokens, while budgeting oversized results is worth several times more. savings.py computes both; report the token figure and say which one a claim rests on.

5. Harness turns are not human turns. Hook feedback and system reminders appear as user records. They match correction patterns like "stop" and inflated a "user corrected the model" metric 8×. Filter on the isMeta field; a startswith("<") heuristic does not catch them.

Classifying intent

scan produces intents.jsonl but assigns no use-case. Run python -m gaia.factory.harvest.classify, which batches the sessions, labels each from its opening instruction against a closed taxonomy, and writes $FACTORY/labels.txt as <8-char-session-prefix> <use-case> — one tag per line, the format report --labels and context --labels read. Then re-run both with --labels.

Three properties matter, and they are enforced rather than advised:

• The taxonomy is closed. A batch that invents tags makes its per-use-case tables incomparable with every other batch.
• Nothing partial is written. A batch whose labels do not cover it exactly is an error; labels.txt appears only once every batch has validated.
• no_instruction is assigned locally. A session whose transcript carries no opening instruction cannot be classified from one, and folding those into other hides them among sessions that did state a goal.

Classify from the first user message, never the auto-generated title — the title summarises what happened, which leaks the outcome into the label.

To label by hand, write the same two-column file yourself. It carries session-id prefixes, so it belongs in the cache directory like everything else derived. report checks those prefixes against the corpus either way: nothing matching is an error, and partial coverage is stated in every use-case table so the labelled subset is never read as the whole corpus.