Knowledge Wiki Merge

Knowledge Wiki Merge

Detect duplicate concept pairs and interactively merge them. Presents one pair at a time — you decide whether to merge, dismiss (never show again), or skip. Merging is destructive and irreversible, so each decision is confirmed before execution.

Steps

1. Establish the working directory

The knowledge base root is the Git repository root. Run git rev-parse --show-toplevel and store the result as KNOWLEDGE_PATH.

Use KNOWLEDGE_PATH for all subsequent steps.

2. Find and filter duplicate candidates

Structural candidates are pairs detected by shared source material — concept files that share two or more ## Sources entries. Run:

node {KNOWLEDGE_PATH}/scripts/wiki/candidates.mjs find-shared-source-concepts

This script automatically filters out previously dismissed pairs from Wiki/.state.json. Output is { "candidates": [...] } sorted by shared source count descending. Tag each as detection: "structural". Do not pipe through head or any other truncating command — every candidate must be evaluated.

LLM pre-filter (structural only): Before proceeding, review the structural candidates and eliminate any pair that is clearly about different topics despite sharing sources — pairs where the shared sources happen to cover two unrelated ideas (e.g. applescript and email-marketing appearing in the same AppleScript email tutorial). For each eliminated pair, call:

node {KNOWLEDGE_PATH}/scripts/wiki/wiki-state.mjs dismiss-pair knowledge-wiki-merge {pathA} {pathB}

where pathA and pathB are the full relative paths (e.g. Wiki/Concepts/applescript.md). Be conservative: only dismiss pairs you are confident are unrelated. A wrongly auto-dismissed pair is hidden from all future runs and requires manually editing Wiki/.state.json to recover.

Semantic candidates are pairs identified by conceptual overlap — synonyms, one being a strict subset of the other, or articles that would naturally be merged — without necessarily sharing sources. Perform this pass by running:

node {KNOWLEDGE_PATH}/scripts/wiki/wiki-index.mjs read-concepts > /tmp/wiki-concepts.md

Then read /tmp/wiki-concepts.md in full using the Read tool, using offset/limit to page through it if the file exceeds the Read tool's line limit. Do not pipe through head or any other truncating command — the concept list may be thousands of lines, and truncating it means missing potential duplicate pairs. Use your judgment to identify semantically overlapping pairs from the complete list. Tag each as detection: "semantic". Pairs found by both structural and semantic methods are tagged detection: "structural+semantic". Skip the LLM pre-filter for semantic candidates — they were already identified by LLM judgment.

If no candidates remain, print No duplicate candidates found. and stop.

3. Present and resolve each candidate

Process one candidate at a time. Use a separate interaction for each pair — never combine multiple pairs into a single question, even if you intend to recommend the same action for several in a row.

For each remaining candidate pair, work through the following sub-steps in order.

3a. Summarize the pair

Read both concept files. Present a brief summary as a markdown table (not in a code block) so it renders:

Candidate pair (shared sources: {N}, detection: {structural | semantic | structural+semantic})

3b. Determine recommended direction and ask what to do

Before asking, determine which concept should be the primary by applying these factors in order:

1. Semantic scope (primary factor): the broader or more general concept is the primary; the narrower or more specific concept merges into it
2. Prose depth: if scope is similar, more paragraphs → likely primary
3. Source count: if prose is similar, more sources → likely primary

Ask the user what to do using exactly four options as follows.

Always include both merge directions. Never drop one because you think it is obviously wrong — the user makes that call.

Never add "(Recommended)" to Dismiss or Skip — not in the label, not in the description, not anywhere. These options are always neutral.

If a clear primary can be determined, put (Recommended) after the recommended merge option and place it first:

If no clear primary can be determined (both concepts are similarly scoped, similarly long, and have similar source counts), state this explicitly. Do not add (Recommended) to any option:

When using an interactive question/options tool, keep the question text short: "What would you like to do with this candidate pair?" Do not cram the summary, reasoning, or option details into one long question line.

The user must still be able to see the full decision context while choosing: the candidate summary, recommendation reasoning, all four options, and the stop instruction. This context may be in the tool's body/details area, or in an immediately preceding assistant message if that message remains expanded and visible while the dialog is open. If the available question tool cannot keep that context visible together with the options, do not use it for this prompt. Instead, render the summary, reasoning, numbered options, and reply instructions as one normal markdown message, then wait for the user's reply. Accept 1, 2, 3, or 4 (or "stop" to halt all remaining pairs).