Debug Run Transcript Maintainer

Debug Run Transcript

Analyze Rudder agent runs by reconstructing the execution story from the best available source.

Purpose

Runs fail for several different reasons:

• the model/runtime emitted an error
• the transcript parser missed useful structure
• the event stream is incomplete
• the stored excerpts are too shallow
• a tool call succeeded, but Rudder's message/run lifecycle marked the workflow failed during stream finalization, persistence, or UI status mapping
• the operator has only a partial run ID or limited context

This skill helps diagnose those cases without getting stuck on the wrong data source.

Debugging proves what happened in a run; it does not by itself prove that a product fix works. When a transcript diagnosis leads to a code, CLI, skill, runtime, or UI change, hand the work back to the lifecycle verification path and require product proof for the affected actor and terminal surface.

Source Priority

Always use sources in this order:

1. Run-intelligence loader/API
   • Best default.
   • Reads run metadata, run events, and the underlying run log.
   • Reconstructs transcript entries with runtime-specific parsers.
2. Filesystem run log fallback
   • Use when the local API is unavailable but run logs exist on disk.
   • Good for transcript/tool-call reconstruction.
3. Direct database queries
   • Use only for targeted checks or when the first two paths are unavailable.
   • DB rows alone are not the full transcript story.

Important Lessons / Known Traps

• Do not assume ~/.rudder/instances/dev/postgres-uri exists. In this repo it is not a reliable universal entrypoint.
• Do not start with heartbeat_run_events and assume they are the complete transcript. They are supplemental run events, not the full parsed execution trace.
• Do not write WHERE id LIKE 'prefix%' against uuid columns. Cast first: id::text ILIKE 'prefix%'.
• Do not assume pnpm exec tsx works from the repo root here. Prefer the repo-local launcher:

node cli/node_modules/tsx/dist/cli.mjs ...

• Do not treat stdout_excerpt / stderr_excerpt as the whole log. They are quick diagnostics only.
• If /api/run-intelligence/runs/<id>/log returns 404, do not assume the run has no raw log anywhere. First check whether you are querying the wrong Rudder instance (for example dev server/API while the run log lives under ~/.rudder/instances/e2e/data/run-logs/...).

Workflow

1. Identify the run

If the user gives:

• a full run ID: use it directly
• a short prefix like 7d28669d: treat it as a prefix
• a recent-run batch request like "prod Z Studio 最近 30 个 run": treat it as batch mode and identify the org/runtime/time window before deep-diving
• only an agent or timeframe: first help locate likely runs before deeper analysis

If the user provides no identifying info at all, ask for at least one of:

• run ID or prefix
• agent name
• approximate time window

1.1 Batch mode for recent runs

Use batch mode when the user asks for recent N runs, org-level run quality, efficiency, repeated failures, automation output quality, or "有什么可以优化".

Batch mode is not the same as a Codex session benchmark. Stay on Rudder agent run evidence: heartbeat_runs, run-intelligence metadata, run logs, result_json, usage_json, stderr/stdout excerpts, and transcript outlines.

Workflow:

1. Resolve the active Rudder instance and org. Prefer explicit org names from the prompt, then live API/org listings, then local instance files.
2. Build the cohort with a stable ordering, usually most recent finished runs for the selected org and optional agent/runtime filter.
3. For each run, capture status, duration, runtime, cost/tokens when present, result shape, stderr/error excerpt, and whether raw log/transcript evidence exists.
4. Classify reusable failure classes: no-op heartbeat, missing context, shallow final answer, repeated tool/runtime failure, excessive cost, blocked environment, stale session continuity, or missing handoff artifact.
5. Deep-dive only the representative runs needed to prove each failure class. Do not parse every full log when metadata already shows the distribution.
6. Output optimization proposals tied to evidence, not a generic agent-quality essay.

If localhost, API, or Postgres access is blocked by sandbox or runtime policy, do not stop. Pivot to filesystem-side evidence:

• ~/.rudder/instances/*/data/run-logs
• local run-intelligence artifacts or log stores
• workspace artifacts referenced by recent runs
• database directory or config files that identify the likely instance
• available JSON summaries, excerpts, and session ids

State which sources were unavailable and label any conclusions that are based on fallback evidence rather than live API/DB reads.

1.2 Separate root cause evidence from fix proof

When the user is debugging a concrete failure that may need a fix, keep two ledgers separate:

• Root cause evidence: run metadata, transcript entries, stdout/stderr, events, source code, config, database rows, or runtime state that explains why the run behaved that way.
• Product proof required after a fix: actor, trigger, system effect, terminal surface, seed/mutation data, screenshots, API readback, or CLI output needed to show the workflow now behaves correctly.

If the debug stage finds the likely fix but the terminal workflow has not been rerun, say fix proof missing instead of calling the issue resolved. For agent-facing bugs, prefer rerunning a disposable agent issue or heartbeat path after implementation rather than only checking stored excerpts or database rows.

1.3 Diagnose finalizer failures separately from tool failures

When the user says a tool failed, or a UI surface shows failed, do not assume the named tool was the root cause. Split the execution into layers:

1. Tool result: did the tool call return isError, stderr, HTTP error, or a malformed payload?
2. Assistant output: did the model produce a final answer, structured result, or continuation after the tool result?
3. Runtime process: did the adapter process exit cleanly, timeout, receive a signal, or lose the session?
4. Rudder finalizer: did stream close, message persistence, run status update, transcript parsing, or result extraction mark the run/message failed?
5. Terminal surface: did Messenger, Issue Detail, run-intelligence, or another UI/API consumer show a failure state that disagrees with lower-level evidence?

If tool results are successful but the UI or run row is failed, classify the root cause as finalizer/status-mapping suspected until proven otherwise. The next source of truth is the server log, stream route, adapter finalization path, run events, message status rows, and UI status mapping, not another inspection of the successful tool payload.