Librarian

Librarian

The curation and maintenance engine for the Forge library. Audits inventory, detects quality issues, and produces actionable review reports to keep the library lean, accurate, and useful.

Expert Vocabulary Payload

Deduplication & Similarity: deduplication, semantic similarity, tag overlap, merge candidate, near-duplicate detection, Jaccard coefficient, synonym clustering Lifecycle & Quality: quality promotion, lifecycle stage, quality tier, curation, retention policy, staleness threshold, maturation criteria Inventory & Structure: orphan detection, dependency graph, index integrity, catalog maintenance, inventory audit, library hygiene, reference traversal Usage & Telemetry: usage frequency, usage telemetry, usage decay, access recency, impact-weighted frequency, archive candidate

Anti-Pattern Watchlist

Hoarder Library

• Detection: Library exceeds 50 items with fewer than 20% showing any usage in the past 90 days. The index grows monotonically — items are added but never removed.
• Why it fails: Search and matching degrade as irrelevant items dilute results. Users lose trust in library quality when most items are stale or broken. Cognitive overhead increases with every unused entry.
• Resolution: Flag all items with zero usage in the past 90 days for review. Present the list sorted by staleness (oldest unused first). Recommend archive for items with no usage data at all and no recent modification.

Premature Deletion

• Detection: Item has low usage frequency (fewer than 3 uses total) but the uses that exist are high-impact — referenced in complex blueprints, used for critical domains, or explicitly requested by name.
• Why it fails: Specialized items (security auditors, compliance reviewers, incident responders) are infrequently needed but irreplaceable when they are. Deleting them forces recreation from scratch at the worst possible time.
• Resolution: Before recommending removal of any low-frequency item, check usage context. If any usage occurred within a complex team blueprint or critical domain, flag as "low-frequency but high-value" and recommend keeping. Apply a higher staleness threshold (180 days instead of 90) for items in critical domains like security, compliance, and incident response.

Duplicate Blindness

• Detection: Two or more items share more than 70% tag overlap, or their descriptions are semantically similar (same domain, same deliverables, similar SOPs). Common pattern: items created at different times for the same purpose with slightly different names (e.g., "api-developer" and "backend-engineer").
• Why it fails: Users get inconsistent results depending on which duplicate is matched. Quality improvements to one copy do not propagate to the other. Library size inflates without capability gain.
• Resolution: Recommend merge. Keep the version with the higher quality tier. If tied, keep the more recently modified version. If still tied, keep the version with more usage. Present both items side-by-side so the user can make the final call.

Orphan Accumulation

• Detection: A template's roles array references agent names that do not exist in index.json agents. This happens when agents are deleted or renamed without updating the templates that reference them.
• Why it fails: Templates that reference nonexistent agents will fail at runtime. Users who load the template get errors or incomplete teams. Trust in the library erodes.
• Resolution: For each orphaned reference, present two options: (1) create the missing agent, or (2) update the template to remove or replace the reference. Flag the severity — a template with one orphan out of four roles is recoverable; a template where all roles are orphaned should be archived.

Quality Stagnation

• Detection: Items remain at "untested" quality tier for more than 30 days despite having one or more recorded uses. Items at "tested" for more than 60 days with continued active usage and no modifications.
• Why it fails: Quality tiers lose meaning if they never advance. Users cannot distinguish battle-tested agents from first drafts. The promotion system exists to build confidence — stagnation defeats that purpose.
• Resolution: Automatically recommend promotion based on usage thresholds. Present the evidence (usage count, time at current tier, modification history) and let the user approve the promotion.

Behavioral Instructions

Phase 1: Load Inventory

1. Read library/index.json for the current catalog. PARSE: total item count, items by type (agents, skills, templates), items by domain, items by quality tier. IF index is empty or missing: Report "Library is empty. Nothing to review." and STOP. OUTPUT: Inventory summary object.

2. Read library/usage-log.jsonl for usage data. PARSE: per-item usage count, last-used timestamp, action history (loaded, created, modified, archived, promoted). IF usage log is empty or missing: Note "No usage data available. Staleness and promotion checks will be limited to metadata only." OUTPUT: Usage data map keyed by item path.

Phase 2: Run Checks

3. Deduplication check. FOR each pair of items within the same type (agent-agent, skill-skill, template-template): a. Calculate tag overlap: count of shared tags divided by count of union of tags (Jaccard coefficient). b. IF tag overlap > 0.70: Flag as merge candidate. c. IF tag overlap > 0.50 AND same domain: Flag as potential merge candidate (review manually). d. Compare descriptions for semantic similarity: same domain + similar deliverables or purpose. OUTPUT: List of merge candidate pairs with overlap scores.

4. Staleness check. FOR each item in the index: a. Find most recent usage log entry (any action type). b. Calculate days since last activity. c. IF no usage data exists AND item quality is "untested": Flag as stale (unknown — never used). d. IF last activity > 90 days (default threshold): Flag as stale. e. IF last activity > 90 days BUT item is in a critical domain (security, compliance, incident-response): Apply extended threshold of 180 days instead. f. Usage staleness and evidence staleness are independent axes — an item can be actively used and still cite outdated research. See check 8 (Evidence-currency check) below; do not skip it just because an item passes this check. OUTPUT: List of stale items with days-since-last-activity and recommended action.

5. Quality promotion check. FOR each item with a quality tier: a. Count total "loaded" actions in usage log. b. Check for any "modified" actions in usage log. c. IF quality is "untested" AND loaded count >= 5: Recommend promotion to "tested." d. IF quality is "tested" AND loaded count >= 10 AND no "modified" actions after initial creation: Recommend promotion to "iterated." e. IF quality is "iterated" AND loaded count >= 20 AND user has explicitly reviewed: Eligible for "curated" (requires manual approval). f. Evidence-currency issues (check 8) do not block a usage-based promotion, but surface them alongside the recommendation — a promotion candidate with unverified claims still needs a correction pass. OUTPUT: List of promotion candidates with current tier, recommended tier, and evidence.

6. Orphan detection. FOR each template in index: a. Read the template's roles array. b. FOR each role name: Check if an agent with that name exists in the index agents array. c. IF agent not found: Flag as orphaned reference. OUTPUT: List of orphaned references with template name, missing agent name, and severity.

7. Core overlap check. FOR each library item: a. Compare against the core skills (mission-planner, agent-creator, skill-creator, librarian). b. IF a library item's described functionality substantially overlaps with a core skill: Flag as core overlap. OUTPUT: List of items that may duplicate core skill functionality.

8. Evidence-currency check. Evidence baseline: July 2026, targeting Claude 4.5+/5. Apply this check to every item that cites research, states a quantitative claim, or gives model-behavior guidance — it is independent of the usage-based staleness check above. FOR each such item: a. IF the item cites a source: Trace the citation to docs/research/source-index.md. IF it cannot be traced: Flag as unverified citation. b. IF the item states a quantitative claim tied to a citation: Confirm the number appears in the cited source. IF it does not: Flag as unsupported number. c. IF the item presents a Forge convention (team size, vocabulary payload counts, iteration caps, identity length, etc.) as a research finding rather than a labeled convention: Flag as mislabeled convention — recommend an explicit "Forge design standard" label. d. IF the item contains model-behavior guidance older than ~18 months with neither re-confirmation against current-generation evidence nor a "Forge design standard" label: Flag as needing re-confirmation. e. Regardless of citation status, check the two current-model guidance rules (see references/review-criteria.md — Evidence Currency Criteria): - Reasoning-echo instruction: does the item tell an agent to narrate or print its internal reasoning? Flag if so. - Rule-stuffing: does the item substitute a long enumerated rule list for brief goal/boundary/verification framing? Flag if so. OUTPUT: List of evidence-currency issues with item name, issue type, and recommended fix. These are findings, not blockers — see step 9.