Palace Index Curator

Palace Index Curator

Overview

The web-research hooks auto-capture every WebFetch and WebSearch into hooks/memory-palace-index.yaml, storing each as a markdown file and an index entry. Captures land at the defaults routing_type: pending, maturity: seedling, importance_score: 50, and nothing advances them. Left alone, the index becomes a write-only graveyard: the majority of entries are never incorporated, analyzed, or surfaced.

This skill drains that backlog and keeps it drained. It wires the capture index to the corpus tooling the plugin already ships (decay_model, keyword_index, marginal_value) through three commands: a read-only report, a dry-run-first promotion engine, and a SessionStart surfacing hook.

When To Use

• A commit was blocked because the index still carries pending entries the drain held back.
• The capture backlog has grown and most entries are still pending.
• You want a corpus health report (inert ratio, orphans, topic clusters).
• You want stored research surfaced automatically during sessions.

When NOT to Use

• Ingesting a single new resource: use knowledge-intake.
• Searching stored knowledge ad hoc: use knowledge-locator.
• Tending a digital garden file: use digital-garden-cultivator.

Workflow

1. Analyze (read-only)

uv run python scripts/memory_palace_cli.py index report

Reports total entries, the inert ratio, orphaned captures (entries whose backing file is gone), the largest topic clusters by domain, and the top promotion candidates. Writes nothing.

2. Incorporate (dry-run, then apply)

# Dry run: prints promote/archive proposals, writes nothing.
uv run python scripts/memory_palace_cli.py index promote

# Apply: backs up the index under data/backups/, then persists.
uv run python scripts/memory_palace_cli.py index promote --apply

Each pending entry is classified into one action:

• promote: recent, authoritative, or clustered. Gets a real importance score, a routing type, and maturity seedling -> growing.
• archive: orphaned or older than the archive horizon and never revisited. Marked archived rather than promoted, following the principle that unused captures should drain, not accumulate.
• hold: everything else stays pending with no change.

Applying is idempotent: promoted and archived entries are no longer pending, so a second run proposes nothing new. The dry-run diff is always shown before --apply writes.

Running these by hand is the exception. --apply runs on every commit from scripts/precommit_palace_maintenance.sh, so the backlog drains continuously rather than in occasional sweeps. Reach for the commands above when a commit is blocked, or when you want the dry-run diff before the hook decides for you.

3. Committed state must be drained

The commit that carries the index must carry it with zero pending entries. scripts/check_capture_index_drained.py runs at the end of the maintenance hook and fails the commit otherwise, and tests/test_capture_index_artifact.py re-checks the same invariant in CI so a bypassed hook does not land a backlog.

Two things make that gate reachable rather than a standing block:

• The capture write stages the index (hooks/shared/deduplication._stage_index). Without it, pre-commit reverts the unstaged write before any hook runs, so the drain reads a tree the fresh capture is missing from and converges on a fixed point that excludes exactly the entries it exists to process. That is how 47 captures accumulated behind a drain that reported nothing to do.
• The drain resolves promote and archive by itself. Only hold survives it, so a blocked commit means a specific capture needs a person to score or archive it. The gate names the keys.

4. Surface (learn)

A SessionStart hook (hooks/index_surfacer.py) names the highest-value promoted captures at the start of a session. It is disabled by default. Enable it in memory-palace-config.yaml:

feature_flags:
  context_injection: true

The hook only speaks when promoted entries clear the importance floor, and it exits silently on any error so it can never block a session.

The corpus keyword index is a separate artifact

The three steps above all operate on the capture index at hooks/memory-palace-index.yaml. Retrieval reads a different file: data/indexes/keyword-index.yaml, built from the staging captures and consumed by cache_lookup. Curating one does nothing to the other.

That keyword index is derived data and is not tracked in git, so a fresh checkout has none at all. Rebuild it with:

# Report what would be indexed, writing nothing.
uv run python scripts/build_indexes.py --dry-run

# Write data/indexes/keyword-index.yaml.
uv run python scripts/build_indexes.py

The builder refuses to write an empty index over a populated one. An empty corpus is reported with "wrote": false and any existing index is left untouched. Writing entries: {} over real data is how the corpus went dark in 1.5.0, and it stayed dark because the regeneration script named in that stub file had never been written.