Issue Handler

Issue Handler — End-to-End Orchestrator

This is the high-level scenario skill for handling a single GitHub issue. It does not do the detailed work itself; it sequences the leaf skills into one iterative pipeline and reports the result. Each stage's mechanics live in its own skill — read and follow that skill when you reach its stage.

Every agent-produced diff is a proposal. This skill commits the staged fix onto a dedicated local branch agent/fix-issue-<N> (single bug) or agent/fix-issue-<N>-<seq>-<slug> (per batch sub-item) and records it in a fix_result*.json. When the bug is owned by an upstream issue — a mirrored pytorch/pytorch#<M> DISABLED test, or a command pointing at an upstream issue — the branch is named after that number instead: agent/fix-pytorch-issue-<M>[-<slug>] (see Branch naming). It never pushes, tags, or opens a PR — the invoking workflow reads each fix_result*.json, exports base_sha..branch as a patch, and a human applies it. The leaves themselves only stage (they never commit).

Contents

• Pipeline overview
• Inputs
• Execution modes
• Branch naming
• Stage 0: Re-run gate
• Stage 1: Triage
• Stage 2: Reproduce
• Stage 1u: Batch-issue fan-out (skip-list + heterogeneous)
• Stage 3: Root cause
• Stage 4: Implement
• Stage 5: Verify
• Stage 6: Report
• Iterative loop bounds
• Issue-body status contract

Pipeline overview

Single-bug path (default for issue_type=single-bug):

triage → reproduce → root-cause → implement → verify → report
                       ↑                          |
                       └─── loop up to 3 times ───┘

Batch path (issue_type=batch-bug, either batch_kind) — fan out the single-bug pipeline per sub-item, each on its own fix branch:

triage → [preflight: install nightly wheel once]
       → for each sub-item:
             reset + reproduce
               ├─ NOT_REPRODUCED → heterogeneous: ALREADY_FIXED · skip-list: STALE_SKIP (+follow-up)
               └─ REPRODUCED → branch per [Branch naming](#branch-naming)
                               → root-cause → implement → verify
             (any failure marks the sub-item and continues the batch)
       → fan-out report

skip-list vs heterogeneous differ only in how a NOT_REPRODUCED sub-item is labeled; the loop is identical.

Inputs

• A GitHub issue on intel/torch-xpu-ops or pytorch/pytorch (URL, number, or raw body).
• pytorch_dir — path to a local pytorch checkout, resolved as described in fix-reproduce Prepare. If absent, this skill lets fix-reproduce / fix-root-cause clone it into $XPU_OPS_ROOT/agent_space_xpu/pytorch/.
• Mode (see below).

Execution modes

The pipeline runs in one of two modes — interactive (default) or pipeline — which changes how every stage reports results and whether it writes to the GitHub issue. Decide the mode at the start and pass it to every leaf. See references/execution-modes.md for the full contract.

• Interactive (default): ask the user when blocked; report conversationally; do not touch the issue body / labels / comments unless the user asks.
• Pipeline (explicit): no human to interrupt — advance the issue's agent:status marker, update stage labels, let leaf skills leave their <!-- agent:<name> --> comments, and stop when the pipeline settles on a terminal verdict.

Branch naming

One rule for both paths: the branch is named after the issue that owns the bug, so a human reading a branch or a patch directory knows which issue a PR closes.

The upstream form applies whenever the sub-item's own identity is an upstream issue — the common case on the CI DISABLED tracking issue, where #N is only a queue and #M is the bug. #M is already unique and stable, so the upstream form carries no seq. When one fix covers several sub-items, the branch is named after the sub-item that was fixed; the rest are recorded in covers.

M is the upstream issue the sub-item resolves to: the pytorch/pytorch#<M> reference on its checklist line, its linked child reference, or the upstream issue named in the invoking comment. If a sub-item has no upstream number, use the agent/fix-issue-<N>-… form.

Stage 0 — Re-run gate (first-run vs re-run)

Pipeline mode only; skip in interactive mode (a human is already driving, so just run the full pipeline). An agent:active issue is frequently re-triggered — a maintainer leaves feedback, or the bot is re-invoked with no new information. This gate decides, before spending a build, whether this is a fresh run, a human-feedback re-run (highest priority), or a bare re-run (skip work that already ran and is still valid).

Step 0.1: Detect prior agent activity

Find the most recent agent comment and its timestamp:

last_agent_ts=$(gh issue view "$N" --repo "$OWNER/$REPO" --json comments \
  --jq '[.comments[] | select(.body | test("<!-- agent:"))] | last | .createdAt // ""')

Empty last_agent_ts → first run. Skip the rest of Stage 0 and go to Stage 1 normally.

Step 0.2: Detect new human feedback

A comment is human feedback when it is authored by a non-bot account and created after last_agent_ts. (The bot account is the one that authored the <!-- agent:* --> comments; exclude it by login.)

new_human=$(gh issue view "$N" --repo "$OWNER/$REPO" --json comments \
  --jq --arg ts "$last_agent_ts" --arg bot "$BOT_LOGIN" \
  '[.comments[] | select(.author.login != $bot and .createdAt > $ts)] | length')

• new_human > 0 → human-feedback re-run. Human feedback is the highest priority signal. Read every such comment verbatim and prepend it to the failure description you hand to Stage 3 (fix-root-cause takes a free-form failure description; a leading "Maintainer feedback since last run: ..." block steers the re-analysis without any new leaf input). Run the full pipeline from Stage 1; do not take any of the skip fast-paths below. A human saying "still wrong" or "change X" overrides any cached verdict.
• new_human == 0 → bare re-run. Continue to Step 0.3.

Step 0.3: Bare re-run — skip what is still valid

No human pointed anything out since the last agent comment, so re-doing the whole pipeline would just repeat identical work. Re-run only the cheap front of the pipeline and compare against last time:

1. Run Stage 1 (triage) and Stage 2 (reproduce) as normal.
2. Compare the reproduce result to the previous run. The previous refined_command + verdict are recoverable from the last <!-- agent:root-cause --> comment's analyzed_sha context, or re-derived by reading the prior <!-- agent:reproduce --> / sweep comment. "Identical" means same verdict and same refined_command.
   • Reproduce differs (now passes, or a different command reproduces) → the situation changed on its own; resume the full pipeline from Stage 3 (root-cause) with the new reproduce result. Do not reuse the cached root-cause.
   • Reproduce identical → nothing observable changed. Hand off to Stage 3, which runs fix-root-cause's own <!-- agent:root-cause --> analyzed_sha fast-path: if target_repo HEAD sha equals the recorded analyzed_sha, that leaf re-emits the prior verdict verbatim and this orchestrator stops (the earlier outcome — fix already staged, or NEEDS_HUMAN — still stands; there is nothing new to do). If the sha moved, fix-root-cause re-analyzes and the pipeline continues from Stage 3 as usual.

This never skips Stage 1/Stage 2 — they are cheap (text + nightly wheel) and are the only way to notice the failure went away. It only avoids the expensive Stage 3-5 rebuild+fix when both the observable failure and the analyzed code are unchanged.

Stage 1 — Triage (issue-triage)

Call issue-triage on the issue body + comments. It emits issue_type (single-bug / batch-bug / nonbug), batch_kind (skip-list / heterogeneous / null), reproduction_missing (yes / no), scope, runtime_dependencies, and a preliminary handling (agent-fixable / needs-human).

Branch on issue_type (triage already made the batch-vs-nonbug call; no re-detection here):

• nonbug → stop the fix pipeline; skip to Stage 6 Report with SKIPPED(reason=nonbug).
• single-bug with reproduction_missing=yes → stop; Stage 6 Report with NEEDS_HUMAN(reason=reproduction_missing). issue-triage's own comment already asks the reporter for a reproducer.
• single-bug with reproduction_missing=no → continue to Stage 2.
• batch-bug → Stage 1u, passing through batch_kind (the loop uses it only to label a NOT_REPRODUCED sub-item).

Stage 2 — Reproduce (fix-reproduce)

Only for the single-bug path (issue_type=single-bug). Call fix-reproduce with:

• reproducer_command — extracted by issue-triage from the issue body.
• stage=nightly — reproduce against the nightly wheel.
• ci_repo — inferred from repo (torch-xpu-ops for issues on intel/torch-xpu-ops, pytorch for pytorch/pytorch), or the value the bot passes explicitly.

Branch on its verdict:

• REPRODUCED → continue to Stage 3. Record the refined_command and base for downstream stages.
• NOT_REPRODUCED → the issue is stale; Stage 6 Report with SKIPPED(reason=no_longer_reproduces).
• NO_REPRODUCER → Stage 6 Report with NEEDS_HUMAN(reason=no_reproducer).
• CANNOT_VERIFY → Stage 6 Report with NEEDS_HUMAN(reason=cannot_verify) and the blocker field.

Stage 1u — Batch-issue fan-out

One loop for every parent issue that tracks multiple children. The parent lists several sub-items; run the single-bug pipeline on each independently, each on its own fix branch, and report every outcome back on the parent. "Fix what you can" — a sub-item that can't be fixed is marked and the batch continues.

Entered for issue_type=batch-bug. issue-triage already set batch_kind; the two kinds share the entire loop and differ only in how a NOT_REPRODUCED sub-item is labeled (see step 2 below):

• heterogeneous — the parent body lists distinct sub-bugs. A NOT_REPRODUCED entry means that bug is ALREADY_FIXED (no longer reproduces); nothing to do.
• skip-list — a Bug Skip issue listing homogeneous already-skipped tests. A NOT_REPRODUCED entry means the test now passes, so its skip decorator is now stale and should be removed (follow-up).

No child GitHub issues are created. Each sub-item is a checklist entry on the parent; its fix lives on a dedicated branch so a human can open one PR per fixed sub-item.

Extract sub-items

Parse the parent body into a list of sub-items. Each is either:

• an inline sub-bug — a checklist line naming a test node id or reproducer (skip-list entries are always this form; normalize a bare Class::method to a node id, fix-reproduce's Prepare step resolves the file), or
• a linked child reference owner/repo#N — fetch that issue and use its body as the sub-item's failure description. Only follow intel/torch-xpu-ops and pytorch/pytorch references; ignore any other repo (untrusted, per fix-root-cause).

A sub-item may also arrive as a plain node id from the caller (a nightly report, an email, a log excerpt): normalize Class::method to a node id, drop blank lines, headers and comments, and collapse exact duplicates.

Split infra failures out before the loop. A runner crash, a docker pull timeout, a full disk or a missing device is not a test failure and will not reproduce. Record these as NEEDS_HUMAN(infra) and report them; they never enter the loop.

Give each sub-item three identifiers used for branch naming:

• seq — 1-based position in the body checklist order. Readable and maps a branch back to its body line. Not the stable identity (editing or reordering the body changes it).
• slug — the stable identity, derived from the sub-item's test node id: take the leaf method name, strip device suffixes (_cpu / _xpu / _cuda / _meta) and any dtype suffix, lowercase, keep [a-z0-9._-], truncate to 40 chars. If two sub-items produce the same slug, append -2, -3, … in body order so every slug is unique within the issue.
• upstream — the pytorch/pytorch#<M> number this sub-item is filed as, from its checklist line or its linked child reference; empty when there is none.

Name the branch per Branch naming: an upstream sub-item gets agent/fix-pytorch-issue-<M>-<slug>, otherwise agent/fix-issue-<N>-<seq>-<slug> (N = parent issue number). On a re-run, match a sub-item to its prior branch by slug — look for an existing agent/fix-*-<slug> branch; if found, that is the same sub-item (rename it if its seq moved, never create a duplicate). Skip headers, prose, and empty lines.

Preflight (many entries): install nightly wheel once

When the list is long (skip-list issues routinely have dozens), front-load the one real wheel install so the per-entry fix-reproduce(stage=nightly) calls each find the env already current and return fast. fix-reproduce always issues pip install --pre --upgrade (it refuses to trust a stale wheel); running it once here does the real work. There is no skip_wheel_install flag — this is purely an ordering optimization:

pip3 install --pre --upgrade torch torchvision torchaudio \
  --index-url https://download.pytorch.org/whl/nightly/xpu
python -c "import torch; print('nightly:', torch.__version__)"

One fix may cover several sub-items

Entries in a batch are often the same bug in the same class. Fixing each one separately runs root-cause and a source build (1-3 h) per entry and produces the same patch several times.

Do not guess which entries share a fix:

1. Order the REPRODUCED entries so likely relatives sit together (same test class or file, same error). Ordering only; it decides nothing.
2. Fix the first entry.
3. Re-run the other entries in that group against the staged fix. The build is already done, so this costs one test run each. fix-verify already produces a before/after table; extend it to the group.
4. Entries that now pass go in the fix's covers list and are not fixed again. Entries that still fail take their own turn from step 2.

One fix gets one branch and one fix_result.json, naming in covers every entry it also fixed.

Per-sub-item loop

Capture the two base SHAs once, then for each sub-item reset both checkouts per the shared reset-between-entries recipe — a prior sub-item's staged diff must not bleed into the next.

For each sub-item:

1. Reset both checkouts to the base SHAs (shared recipe).

2. Reproduce (fix-reproduce). It detaches the pytorch tree to its base (origin/main or the ci_commit fallback) and returns that base plus a refined_command.

   stage follows the issue, like allow_skip:

   • CI break (pytorch-ci-failure, or a mirrored DISABLED test) → stage=auto. Many CI failures only reproduce on a source build. stage=nightly returns NOT_REPRODUCED as soon as the nightly wheel passes and does not fall through, which records unfixed work as already fixed.
   • Everything else → stage=nightly.

   Branch on the verdict:

   • REPRODUCED → continue to step 3; keep its base and refined_command.
   • NOT_REPRODUCED → nothing to fix; record it and go to the next sub-item. The label depends on batch_kind:
     • heterogeneous → ALREADY_FIXED: the reported bug no longer reproduces on latest nightly (upstream fixed it, or it was flaky). No action needed.
     • skip-list → STALE_SKIP: the test now passes, so its skip decorator is obsolete. Record a follow-up to remove the decorator — the orchestrator does not delete it here (see "Stale skips" below).
   • NO_REPRODUCER → INVALID_ENTRY (renamed/removed test, or malformed). Record, continue.
   • CANNOT_VERIFY → UNVERIFIED (environmental). Record, continue.

3. Root-cause, then branch. Run Stage 3 (fix-root-cause) first — it returns target_repo, which decides which checkout the fix (and thus the branch) lives in:

   • target_repo=pytorch → target_repo_dir = pytorch_dir, base is the reproduce base on the pytorch tree.
   • target_repo=torch-xpu-ops → target_repo_dir = pytorch_dir/third_party/torch-xpu-ops, base is xpu_ops_base (the submodule's pinned commit from the shared recipe). fix-implement / fix-verify handle the xpu.txt pin rewrite internally; the orchestrator only creates the branch.

   Create the isolated fix branch on target_repo_dir off its base so the diff is pushable on its own (Branch naming):

   # upstream sub-item: branch="agent/fix-pytorch-issue-${M}-${slug}"
   branch="agent/fix-issue-${N}-${seq}-${slug}"
   git -C "$target_repo_dir" checkout -B "$branch" "$base"

   Then run Stage 4 → 5 (same contract and 3-attempt bound as the single-bug path). fix-implement expects exactly this: a fresh branch the orchestrator just created, clean worktree.

4. On any leaf NEEDS_HUMAN / CANNOT_VERIFY / FAILED (or attempts exhausted): mark this sub-item blocked with the reason, record it, and continue to the next sub-item — never abort the whole batch on one hard sub-item.

5. On fix-verify PASSED: do NOT push or open a PR. Update the sub-item's fix_result-${slug}.json in place (written at verdict=PENDING_VERIFY when the branch was committed) to verdict=PASSED with fix_repo_dir, branch, base_sha (the sub-item's base from step 3), changed_files, so the workflow can export base_sha..branch per sub-item. As in the single-bug path, commit the branch and write the PENDING_VERIFY record as soon as fix-implement returns READY, not only on PASSED, so a crash mid-sub-item leaves a recoverable record instead of an orphan branch.