Analyze Transcript

Analyze Agent Transcript

Download and analyze fullsend agent JSONL transcripts from GitHub Actions runs.

Prerequisites

• gh CLI authenticated with access to the target repository
• Python 3.8+

Rules

• Never auto-dispatch subagents. Try search, errors, or conversation --lines first. Only suggest a subagent if targeted commands aren't enough — never spawn one silently.
• Always use relative paths. Pass .transcripts/run-<id>/... to the script, not absolute paths. Absolute paths trigger permission prompts.
• No shell variables. Never assign paths to variables or export them. No DLDIR=..., TRANSCRIPT=..., SANDBOX_LOG=..., BASE_DIR=..., export .... Inline the literal path in every command. This applies to the download dir, transcript paths, sandbox log paths, and the script path. Variables cause commands to drift from the allowed-tools patterns and trigger unnecessary permission prompts.
• Limit output. Use | head -N, | tail -N, or --lines to keep transcript output short. Never dump a full conversation into the main context.
• Plain find only. Never use -exec, -ls, or -printf with find. Use find ... -print (the default). If you need file sizes, run ls -lh on the directory instead.

Script location

The analyzer script lives alongside this skill:

skills/analyze-transcript/analyze-transcript.py

Resolve the path from the skill's base directory. If this skill was loaded from a base directory, the script is at <base-dir>/analyze-transcript.py.

Question routing

Route the user's question to the right subcommand before reaching for full conversation output:

Workflow

1. Get the run

The user provides a GitHub Actions run URL. Extract owner, repo, and run ID.

URL formats:

• https://github.com/OWNER/REPO/actions/runs/RUN_ID
• https://github.com/OWNER/REPO/actions/runs/RUN_ID/job/JOB_ID
• https://github.com/OWNER/REPO/actions/runs/RUN_ID/attempts/N

2. Check run status

gh run view <run-id> --repo OWNER/REPO --json status,conclusion,createdAt,updatedAt

If still running, tell the user and offer to watch it.

3. Download artifacts

mkdir -p .transcripts/run-<run-id>
gh run download <run-id> --repo OWNER/REPO --dir .transcripts/run-<run-id>

The .transcripts/ directory is gitignored. Using the working directory avoids permission prompts for /tmp writes.

4. Find artifacts

Downloaded artifacts have this structure:

.transcripts/run-<run-id>/fullsend-<agent>/
  agent-<type>-<id>/
    logs/
      openshell-sandbox.log    # OCSF network/process/sandbox events
      openshell-gateway.log    # Gateway-side events
    iteration-1/
      transcripts/
        <agent>-<session-id>.jsonl   # Main agent transcript (Claude Code)
        <agent>-agent-*.jsonl        # Subagent transcripts (Claude Code)
        <agent>-<timestamp>_<id>.jsonl  # pi session file (runtime: pi); same subcommands apply

Find transcripts and logs:

find .transcripts/run-<run-id> -name "*.jsonl" -type f -print
find .transcripts/run-<run-id> -name "openshell-sandbox.log" -type f -print

Each JSONL file is one agent session (main agent or subagent). The sandbox log contains all OCSF network events for the run. If you need file sizes, use ls -lh on the directory.

5. No transcript found?

If find returns no .jsonl transcript files, the run likely failed before the agent started (security scan, infra failure, OPA denial). Fall back to:

1. Check for non-transcript files in the artifact dir:
   • run-summary.json — structured run metadata
   • run-telemetry.jsonl — OTLP telemetry (not a Claude transcript)
2. Check sandbox logs for early failures:

   python3 <base-dir>/analyze-transcript.py netsearch "DENIED" <sandbox-log>

3. Pull GitHub Actions logs directly:

   gh run view <run-id> --repo OWNER/REPO --log-failed

Do not confuse run-telemetry.jsonl with Claude transcripts. Telemetry files contain OTLP spans, not conversation messages. The script will detect this and warn you.

6. Run analysis

Quick audit (summary + errors + tools in one shot):

python3 <base-dir>/analyze-transcript.py audit <path-to-jsonl>

Summary only:

python3 <base-dir>/analyze-transcript.py summary <path-to-jsonl>

Shows: agent type, model, duration, message count, token usage, tool call breakdown, and stop reasons.

7. Deeper analysis based on user questions

Use the appropriate subcommand:

Full conversation flow (alias: conv):

python3 <base-dir>/analyze-transcript.py conversation <path> | head -100

Shows assistant text, tool calls, and results with line numbers. Use | head -N for the start or | tail -N for the end of large transcripts. Use --max-width 0 for full untruncated output.

Note: System prompts and pre-conversation context are not included in conversation output. If the user asks "why did the agent do X", search for decision-relevant keywords first, then show surrounding conversation lines.

Tool usage breakdown:

python3 <base-dir>/analyze-transcript.py tools <path>

Errors and failures only:

python3 <base-dir>/analyze-transcript.py errors <path>

Finds tool errors, permission denials, failed commands, and error mentions.

Search for specific content:

python3 <base-dir>/analyze-transcript.py search "yarn install" <path>

Regex search across all tool results and assistant text.

Restrict to line range (place after subcommand; transcript subcommands only):

python3 <base-dir>/analyze-transcript.py conversation <path> --lines 50-100

JSON output (available on summary, tools, and network):

python3 <base-dir>/analyze-transcript.py summary <path> --json