Vulnhunter Fix

VulnFix — Automated Security Remediation via TDD

Overview

VulnFix takes VulnHunter scan results and automates the full remediation lifecycle:

1. Parse findings from the report (or from vulnhunter-labeled issues in in-place mode)
2. Plan fix order and approach
3. For each vulnerability: write exploit demo → write failing test → implement fix → verify test passes
4. Deliver as PRs — to a private fork (fork mode) or to the same repo (in-place mode)

Modes

The skill runs in one of two modes, dispatched automatically.

Mode dispatch (mandatory)

Before any other action, decide the mode. Run the canonical dispatcher script and parse its stdout:

# Returns one of: mode=in_place / mode=fork / mode=ambiguous / mode=none
# Exit 0 for in_place/fork/none; exit 2 for ambiguous (caller must resolve).
DISPATCH="$(bash "${SKILL_DIR}/scripts/detect_mode.sh" \
    "${TARGET_REPO:-}" "${RESULTS_PATH:-}")"
echo "$DISPATCH"

The script is the source of truth for the dispatch rule — it has its own tests under tests/test_detect_mode_sh.py. Behavior summary:

Parse the mode=... line and the owner_repo=/root=/target=/results= fields with shell expansion or awk. Export them for downstream phases — every later gh call needs $OWNER_REPO, and every reference to the project root uses $ROOT (fork mode falls back to pwd / CWD).

When MODE resolves to in-place, the rest of the workflow uses CWD as the target repo, treats <CWD>/.vulnhunter-fix/ as the work directory, and skips Steps 2 (TARGET_REPO/RESULTS_PATH prompting) and 5 (clone into ./work/) below.

RESULTS_PATH in in-place mode (local scan reports)

When the operator supplies RESULTS_PATH in in-place mode, detect_mode.sh emits mode=in_place results=<path> — a single mode, with the findings source riding as an optional field. prompts/parse_issues.md § Step 0 reads that field and skips the GitHub-issue harvest, sourcing findings from disk instead. It writes no_source_issues: true + results_path to work.json. Downstream renderers (deliver.md, templates/pr_body_cluster.md) branch on no_source_issues to switch PR-body language from Closes #N to Addresses VULN-N per <results_path>.

Findings-source is a property of the work, not a property of the mode.

Handling mode=ambiguous

The user is inside a GitHub checkout AND supplied fork-mode arguments — both signals are present and only one can be the intended target. Use AskUserQuestion to disambiguate with these two options (label / description):

• "In-place — fix <owner_repo> (current checkout)" — discard $TARGET_REPO/$RESULTS_PATH, re-run dispatch with empty args so the result is mode=in_place.
• "Fork mode — fix <target> using <results>" — cd out of the current checkout (or set OWNER_REPO="") and re-run with the original args so the result is mode=fork.

Do not guess; the developer's intent is the only signal. Until they answer, do not advance to Step 1.

Handling mode=none

Neither signal was supplied — not in a git repo and no TARGET_REPO/RESULTS_PATH arguments. Use AskUserQuestion to collect both inputs:

• "TARGET_REPO" — full GitHub URL (e.g., https://github.com/org/repo)
• "RESULTS_PATH" — local directory path OR a GitHub URL to a results repo

Once both are collected, re-invoke detect_mode.sh with them as positional args (bash detect_mode.sh "$TARGET_REPO" "$RESULTS_PATH") and proceed on the resulting mode=fork. If the user can't provide one or both, stop with: "Cannot proceed without TARGET_REPO and RESULTS_PATH when run outside a GitHub checkout."

Required Inputs (fork mode only)

• TARGET_REPO: GitHub repo URL (e.g., https://github.com/org/repo)
• RESULTS_PATH: Path to VulnHunter results — either:
  • Local directory: /path/to/RepoName_VULNHUNT_RESULTS_*/
  • GitHub repo URL: https://github.com/org/vuln-findings (will be cloned)

In-place mode harvests findings from vulnhunter-labeled issues; no inputs are required beyond the user running the skill from the target repo's working tree.

Mandatory First Actions

Step 0: Confirm you're running on Opus. (in-place / interactive mode only — headless skips this)

The reasoning load in this skill — clustering findings by topic, the CANNOT_AUTO_FIX collaboration loop, fix synthesis when the report leaves gaps — is calibrated for Opus. Sonnet and Haiku produce noticeably worse results: weaker clustering, hand-wavier fix proposals, more dead-end iterations in the loop.

Check your own model identity from your session system prompt (it says "You are powered by the model named …"):

This is interactive-mode only. Headless mode invokes the executor with a fixed Opus model under the hood and skips this gate.

The check applies even if the user has just switched models mid-session — you may have started this turn on Sonnet because that was the session default before they ran /vulnhunter-fix. The /model command takes effect on the next turn, so stopping here gives the user the clean handoff.

Step 0b: Confirm the installed skill is at upstream main. (in-place / interactive mode only)

The skill itself is versioned in https://github.com/capitalone/vulnhunter on main. Stale installs miss bug fixes (today the failure list includes: TLS-quirk handoffs, sandbox-friendly clone flags, the cluster-as-PR semantic, the --repo rule). Before doing anything else, confirm the user is running the latest skill:

# 1. Get the latest upstream HEAD SHA of main.
UPSTREAM_HEAD="$(gh api repos/capitalone/vulnhunter/branches/main --jq .commit.sha 2>/dev/null)"

# 2. Get the SHA the installed skill is at. The installer (install.sh) writes
#    this on every install. If the file is missing, the user installed by
#    a different mechanism and we can't compare.
INSTALLED_SHA_FILE="${SKILL_DIR}/.installed-from"
if [ -f "$INSTALLED_SHA_FILE" ]; then
    INSTALLED_SHA="$(cat "$INSTALLED_SHA_FILE")"
else
    INSTALLED_SHA=""
fi

The check is a courtesy gate, not a hard one — except when SHAs are confirmed to differ, in which case the user is running a known-stale skill and continuing risks reproducing already-fixed bugs.

Step 1: Run preflight check.

python3 "${SKILL_DIR}/scripts/preflight.py"

If any check fails, stop and report the failure to the user. Preflight only validates LOCAL state (tool versions, disk, working-tree-cleanness, etc.) — it does not check network or gh auth because Python subprocesses can't reliably reach GitHub in some target environments.

Step 1b: Verify gh auth via Bash.

gh auth status >&2 || { echo "Not authenticated. Run: gh auth login" >&2; exit 1; }
gh api user --jq .login >/dev/null || { echo "gh token invalid or GitHub unreachable" >&2; exit 1; }

This runs through the Bash tool's working context (which has TLS the Python preflight doesn't). For in-place mode also verify the user can see origin (using the OWNER_REPO computed in mode dispatch so the call doesn't depend on gh repo set-default):

gh repo view "$OWNER_REPO" --json name,owner >/dev/null \
    || { echo "Cannot view $OWNER_REPO via gh" >&2; exit 1; }

git + gh failure policy — STOP and ask the user

This rule overrides any local instinct to retry, fall back to a different tool, or work around a git / gh failure. Read it before every phase that runs either command.

Some target environments (notably macOS with corporate keychain interception or restrictive sandboxes) make git and gh calls fail intermittently from within Claude Code's Bash tool — even though the same command works perfectly in the user's own terminal. The failure modes vary:

• TLS errors: tls: failed to verify certificate, x509: OSStatus -…, SSL certificate problem: unable to get local issuer certificate, Could not resolve host: api.github.com.
• Sandbox / filesystem denials during git clone: Operation not permitted when copying hook templates into .git/hooks/, fatal: cannot copy '…/commit-msg.sample' to '…/.git/hooks/…'.
• Transient transport errors: Post "https://api.github.com/graphql": …, dial tcp …: i/o timeout.
• Any git or gh invocation exiting non-zero with output that doesn't match a documented, locally-handled case (e.g., git symbolic-ref returning non-zero is expected — your script has a fallback chain; that doesn't trigger this rule. A git clone exit of 128 with sandbox or TLS output does).

When you hit one of these, this is the only acceptable sequence:

1. STOP. Do not retry the same command. Do not retry with different flags. Do not invoke a different but equivalent tool — that is the worst trap. Examples of forbidden "fixes":

   • git clone fails → trying gh repo clone instead.
   • gh issue list fails → trying git ls-remote or curl https://api.github.com/....
   • git clone <url> <repo-relative-path> fails on sandbox → trying git clone <url> $TMPDIR/....
   • Falling through to a different gh subcommand hoping it uses a different transport.

   Every "fix" in this category makes the transcript noisier, wastes the user's time, and never addresses the environmental root cause. The TLS / keychain / sandbox quirks are local to this Bash-tool environment; no command variation will fix them.

2. Tell the user, in plain text, exactly ONE command to run in their own terminal (NOT a multi-command pipeline, NOT a cd && cmd chain — one literal command they can copy and paste). Use this exact format so the command stands out visually instead of getting buried in prose:

   ⚠️ `<original tool call>` failed inside Claude Code's sandbox.
   
   **▶ Run this in your own terminal, then paste the output back (or just "done"):**
   
   ```bash
   <single command, exactly as you would have run it>
   ```
   
   _Why this works: <brief reason — TLS / sandbox / etc.>_

   Mandatory formatting elements (don't omit any):

   • ⚠️ marker on the first line so the user's eye lands there immediately.
   • Bold call-to-action line right above the command — not buried at the end of a paragraph.
   • Triple-backtick fenced block with bash language hint — renders as a distinct monospace box, NOT as 4-space indent (which gets visually mistaken for regular indented text).
   • Blank lines above and below the fenced block for breathing room.
   • The "why" line goes AFTER the command (in italics, smaller-feeling), so the command itself is the visual centerpiece.

   Do not wrap the command across multiple lines with backslash continuations — the user has to copy/paste it as-is, and multi-line bash is error-prone in pasting. If the command is long, accept the long single line.

3. Wait for the user to paste the output. Do NOT continue with any further git or gh calls in the meantime.

4. Use the pasted output verbatim. If the next step in the workflow needed a JSON blob, treat the paste as that blob. If the user was asked to clone a repo, always specify a path inside the project's own .vulnhunter-fix/ work dir (e.g., <project_root>/.vulnhunter-fix/publish/repo). Never instruct the user to clone to $HOME, $TMPDIR, or any other path outside the project — those need extra permission grants for every later access and leave leftover state that breaks re-runs.

5. Repeat for the next call. One command at a time. Never batch.

Exceptions — when you MAY retry or work around:

• The user explicitly says "try a different approach" or "go ahead and retry".
• The failure is a real API error with a structured body — e.g., gh returns an HTTP 4xx with a JSON {"message": "Not Found"}, or git returns a clear domain error like fatal: branch '…' does not exist. Those are actionable in-flow: re-prompt the user, change the branch, etc.

This rule applies in every phase — parse, plan, implement, verify, deliver. If you find yourself running a second git or gh command of the same conceptual kind (clone, fetch, list-issues, …) with no user input in between, you are violating this rule.

Hand-off command templates per language / tool

When the failing command involves a language toolchain that fetches dependencies, the user-paste command MUST include the corporate-proxy env vars — otherwise the user's go test / npm install / pip install will hit the public registry and fail the same way the sandbox call did. The required prefixes:

For Go specifically — this is the most common case in the current target environments — the full template for module-cache repopulation is:

⚠️ `go mod download` failed inside Claude Code's sandbox (network/proxy block).

**▶ Run this in your own terminal, then paste "done" back:**

```bash
cd <ABSOLUTE_WORKTREE_PATH> && GO111MODULE=on GOPROXY=https://<your-go-module-proxy> GOAUTH=netrc go mod download all
```

_Why this works: your terminal can reach your corporate Go module proxy via netrc auth; the sandbox can't. Once the modules land in the shared `~/go/pkg/mod` cache, my sandboxed `go test` calls read from there without needing the network._

Always include the env-var prefix on the same line as go mod — Go does not auto-pick these up from a shell rc file in a non-interactive run, and missing one fails silently or talks to proxy.golang.org which is sandbox-blocked. If you find yourself emitting a bare go mod download to the user, you've forgotten this rule.

Two additional rules every gh call must follow (so user-pasted commands work identically):

1. Always pass --repo "$OWNER_REPO" (or --repo "<owner>/<repo>"). Otherwise gh falls through to gh repo set-default, which most users haven't configured, and the command fails with No default remote repository has been set even when authenticated. Local commands like gh auth status and gh api user don't need a repo.
2. Prefer git over gh for anything that's local-discoverable. Examples:
   • Default branch → git symbolic-ref refs/remotes/origin/HEAD | sed 's|^refs/remotes/origin/||' (no network).
   • Origin URL → git remote get-url origin (no network).
   • Listing branches → git branch --list (no network).
   • Use gh only when the data genuinely lives on GitHub (issue bodies, PR status, label state, etc.).

Step 2: Validate inputs.

In fork mode, both TARGET_REPO and RESULTS_PATH must be provided. If missing, ask the user:

• "Which GitHub repo should I fix?" (for TARGET_REPO)
• "Where are the VulnHunter results?" (for RESULTS_PATH)

In in-place mode, skip this step — inputs come from vulnhunter-labeled issues on origin.

Step 3: Load configuration.

Read config.json from this skill's directory for settings (GitHub host, branch prefix, behavior flags).

Step 4: Set up work directory.

In fork mode: clone the fork into ./work/<fork-repo-name>/ (e.g., ./work/vulnhunter-fix-my-service-api/). The clone is deleted after successful delivery.

In in-place mode: CWD is already the target repo. Create the work tree at <repo_root>/.vulnhunter-fix/, add it to .git/info/exclude (never to tracked .gitignore), and git worktree prune to clean up orphans from previous crashed runs. One git worktree per cluster gets created later by scripts/setup_worktree.sh during Phase 3.