Refresh Index

Refresh Index

Re-ingest every enabled repository in data/skill-index-resources.json so data/skill-index/{owner}_{repo}.json matches upstream. Classify each repo as updated / unchanged / failed / skipped, verify the catalog rebuilds, then open one data-only PR after explicit confirmation.

This is the inverse of skill-index-updater (that skill adds repos). Keep SKILL.md as the spine and load step detail from references/ so the agent's context budget stays small.

When to Use

• User asks to "refresh the index", "update the indexed skills", "sync the catalog", "re-ingest all repos", or "batch-maintain the skill index"
• A scheduled refresh is due, or a release needs current upstream skill metadata

Do not trigger for: adding a new repository (skill-index-updater), authoring or improving a single skill (skill-creator), opening an upstream PR (skill-upstream-pr), or installing/updating skills on the local machine (asm install, asm update).

Repo Sync Before Edits (mandatory)

Before re-ingesting anything, pull the latest remote branch:

branch="$(git rev-parse --abbrev-ref HEAD)"
dirty=0
if [ -n "$(git status --porcelain)" ]; then
  git stash push -u -m "pre-refresh-index: ${branch}"
  dirty=1
fi
git fetch origin
git pull --rebase origin "$branch"
if [ "$dirty" -eq 1 ]; then
  git stash pop || {
    echo "✗ Stash pop failed — recover with: git stash list && git stash show -p stash@{0}"
    exit 1
  }
fi

If origin is missing or rebase conflicts occur, stop and ask the user before continuing. Never silently overwrite local edits to data/skill-index/ or data/skill-index-resources.json.

Prerequisites

Verify each before any ingest. Stop and tell the user if any fails.

• node on PATH (command -v node) — Node >= 22 per package.json engines
• npm on PATH (command -v npm) — required by npm run preindex; the ingester evaluates skills in-process, so no asm binary is needed
• gh on PATH and authenticated (gh auth status) — required for PR creation
• git on PATH and inside the ASM repo working tree (git rev-parse --show-toplevel)
• Network access to github.com (each ingest clones the upstream repo)

Pipeline

Follow these steps in order. Each step has a verification check — do not proceed if the check fails. Long scripts and templates live in references/; read the linked file when you reach that step.

Many agent hosts start each command in a fresh shell. Start every code block with ROOT="$(git rev-parse --show-toplevel)"; RES="$ROOT/data/skill-index-resources.json" instead of relying on variables from an earlier step.

Step 1: Enumerate the index

Read data/skill-index-resources.json and split entries, keyed on the source string (github:owner/repo) because that is the identifier preindex echoes:

ROOT="$(git rev-parse --show-toplevel)"
RES="$ROOT/data/skill-index-resources.json"
jq -r '.repos[] | select(.enabled == true)  | .source' "$RES"
jq -r '.repos[] | select(.enabled == false) | .source' "$RES"
jq -r '.repos[] | select(.enabled == true) | "\(.source)\t\(.owner)_\(.repo)"' "$RES"
jq empty "$ROOT"/data/skill-index/*.json   # exits non-zero if any file is invalid
git -C "$ROOT" status --porcelain -- website/ data/

• enabled[] — "enabled": true. These will be refreshed.
• disabled[] — "enabled": false. These land in skipped with reason "disabled in skill-index-resources.json". The list may be empty; then W = 0.

Verification: enabled[] is non-empty, every per-repo JSON parses, and git status --porcelain -- website/ data/ prints nothing. If any check fails, stop. A clean website/ is what lets Step 5 restore the tracked files build-catalog rewrites.

Step 2: Snapshot pre-run skill counts

Record current skillCount for each enabled repo so Step 7 can report deltas. Run the loop in references/snapshot.md. Missing index file → pre-count 0.

Step 3: Run npm run preindex

Re-ingest every enabled repo. Capture stdout and the exit code — preindex exits 1 if any repo fails, but do not abort. Partial results still classify.

preindex writes each ingest to getIndexDir() before copying it into data/skill-index/. Without an override that is the user's real ~/.config/agent-skill-manager/skill-index/, a change no git diff shows. Always set ASM_CONFIG_DIR to a scratch directory:

ROOT="$(git rev-parse --show-toplevel)"
mkdir -p /tmp/refresh-index/asm-config
LOG="/tmp/refresh-index/preindex.log"
cd "$ROOT"
ASM_CONFIG_DIR=/tmp/refresh-index/asm-config npm run preindex > "$LOG" 2>&1
PREINDEX_EXIT=$?
cat "$LOG"

preindex clones every enabled repo in sequence and can run for many minutes. If the host's command timeout is shorter, run the command in the background and wait until the log ends with the Done: N succeeded, M failed line; then read the exit code.

Verification: the log exists and contains one line per enabled repo: {source} ... {N} skills or {source} ... FAILED: {error}. If the log is empty or no line matches, stop — that is environmental, not a per-repo failure.

Step 4: Classify each repo

Match each {source} log line to data/skill-index/{owner}_{repo}.json in git status --porcelain (which also lists new, untracked index files). Every ingest rewrites timestamps, so "changed" means changed after the timestamps are stripped. Read references/classify.md for the comparison and the four-row signal table. Capture post-run skillCount; failed repos keep the pre-count.

Step 5: Rebuild the website catalog (verification only)

Confirm the refreshed index is structurally valid. website/catalog.json is gitignored — never stage it.

cd "$ROOT"
npx tsx scripts/build-catalog.ts
jq empty website/catalog.json
git restore -- website/ data/repo-stars.json

Verification: the first two commands exit 0. If either fails, stop — the data files are internally inconsistent and the PR must not land. build-catalog also rewrites tracked outputs (website/*-stats.json, website/robots.txt, and data/repo-stars.json on any networked run) with fresh values; git restore -- website/ data/repo-stars.json reverts them. That is safe only because Step 1 confirmed website/ and data/ were clean. It does not touch the gitignored website/catalog.json.

Step 6: Detect unexpected diff scope

Confirm the only files that changed are under data/skill-index/ (and data/skill-index-resources.json only if the user explicitly bumped updatedAt):

UNEXPECTED=$(git status --porcelain | cut -c4- \
  | grep -v -E '^data/skill-index/' \
  | grep -v -E '^data/skill-index-resources\.json$' \
  || true)
if [ -n "$UNEXPECTED" ]; then
  echo "⚠ Unexpected files in diff:"
  printf '%s\n' "$UNEXPECTED"
fi

npm run preindex does not modify data/skill-index-resources.json. If unexpected files appear, stop. Do not commit a mixed change.

Step 7: Print the four-bucket summary

Render the markdown in references/summary-template.md. If X + Y + Z + W does not equal len(enabled) + len(disabled), stop and re-check Step 4.

Step 8: Confirmation gate, commit, and PR

Do not proceed without explicit user confirmation (yes only). Never commit on main: if the current branch is main, create chore/refresh-index-<YYYYMMDD> first. Read references/commit-and-pr.md for the branch step, the diff-stat prompt, conventional-commit message, and gh pr create body. Stage only data/skill-index/ (plus the resources file if intentionally modified). Never stage website/catalog.json.

Verification: gh pr view --json url returns the new PR URL. Print it back to the user.