Concept Synthesis

concept-synthesis — From Raw Stubs to Intellectual Map

Convention: see conventions/quality.md for back-link enforcement and quote-fidelity requirements.

Convention: see _brain-filing-rules.md — output files under concepts/ per the primary-subject rule.

What this solves

Many ingestion pipelines (signal-detector, idea-ingest, voice-note-ingest) create a concept page for every idea mentioned. Over months this produces:

• Thousands of stub pages, many duplicates or near-duplicates
• Timeline entries that repeat the same source across multiple concept pages
• No synthesis — just "the user mentioned X on this date"
• No tier assignments — everything flat
• No clustering — related ideas aren't linked

This skill transforms that raw material into a curated intellectual map.

Architecture

Phase 1: Dedup + merge (deterministic)
  N stubs → ~N/4 canonical concepts
    ├── Jaccard dedup (word-overlap on titles + first-paragraph)
    ├── Substring dedup ("founder mode" vs "founder mode vs manager mode")
    ├── Semantic dedup (LLM: "are these the same idea?")
    └── Merge timelines + aliases from duplicates into the canonical page

Phase 2: Score + tier (deterministic + heuristic)
  Each canonical concept → scored and tiered
    ├── Frequency: distinct sources referencing this concept
    ├── Timespan: first mention → last mention in days
    ├── Breadth: distinct months it appears in
    ├── Engagement: avg engagement on concept-bearing sources (if available)
    └── Tier: T1 Canon | T2 Developing | T3 Speculative | T4 Riff

Phase 3: Synthesize (LLM, T1+T2 only)
  T1 + T2 concepts → rich synthesis
    ├── Evolution narrative: how the idea sharpened over time
    ├── Best articulation: highest-engagement or most precise quote
    ├── Related concepts: cross-links to other concepts
    ├── Context: what was happening when this idea emerged / evolved
    └── Counter-positions: what this idea argues against

Phase 4: Cluster + map (LLM)
  All tiered concepts → intellectual clusters
    ├── Group related concepts into domains (auto-named via LLM)
    ├── Generate cluster summary pages
    ├── Build a master concepts/README.md with the full map
    └── Identify idea genealogies (concept A → evolved into concept B)

Phase 5: Curation cull (rubric + reversible merge)
  Each concept → hard verdict: ELITE | KEEP | MERGE/REWRITE | DELETE
    ├── 6-axis rubric (substance 2x, packaging 1x) + minimum substance gate
    ├── Grounding labels (VERIFIED / OPINION / NEEDS_SOURCE / UNSAFE)
    ├── Cluster budgets + reputational-risk gate
    ├── Merge-with-backlinks into cluster canonicals (fully reversible)
    └── merge_count / independent_sources → emergent tier promotion

Invocation

The skill is markdown agent instructions. The agent uses gbrain's existing operations + LLM passes:

# 1. List all concept pages
gbrain query "type:concept" --limit 10000 --json

# 2. Phase 1 dedup — agent applies Jaccard + substring locally,
#    then LLM passes to identify semantic duplicates.

# 3. Phase 2 tier — agent scores each canonical concept based on
#    frequency / timespan / breadth and writes tier into frontmatter.

# 4. Phase 3 synthesis — for each T1/T2, agent reads the timeline
#    + associated source pages and writes a synthesis section
#    onto the concept page via put_page.

# 5. Phase 4 clustering — agent reads the tiered concept list
#    and writes concepts/README.md with the full intellectual map.

Output: concept page format (post-synthesis)

T1 Canon — full synthesis

---
title: "concept name"
type: concept
tier: 1
tier_label: "Canon"
mention_count: 18
distinct_months: 8
first_mention: "YYYY-MM-DD"
last_mention: "YYYY-MM-DD"
composite_score: 78.4
aliases: ["alternate phrasing 1", "alternate phrasing 2"]
related: ["sibling-concept-1", "sibling-concept-2"]
---

# concept name

**Tier 1 — Canon** | 18 mentions across 8 months

## Synthesis

[2-4 paragraph narrative tracing how the idea evolved, what it means in
the user's worldview, why it matters. Third-person analytical voice.]

## Best Articulation

> "Verbatim quote from a source — the most precise or highest-engagement
> expression of this idea." — [Date](source-url)

## Evolution

| Period | Expression | Signal |
|--------|-----------|--------|
| YYYY-MM | "First articulation" | First use — aspiration frame |
| YYYY-MM | "Sharpening" | Anti-pattern emerges |
| YYYY-MM | "Peak form" | Cleanest expression |

## Related Concepts
- [sibling concept](sibling-concept.md) — relationship description
- [sibling concept](sibling-concept.md) — relationship description

## Timeline
[Full timeline with deduped entries, quotes, source links]

T3 / T4 — stub only (no LLM synthesis)

---
title: "concept name"
type: concept
tier: 4
tier_label: "Riff"
mention_count: 1
---

# concept name

**Tier 4 — Riff** | 1 mention

> "Quote from the source" — [Date](URL)

Output: cluster map at concepts/README.md

# Intellectual Universe

## Canon (T1) — N concepts
The permanent intellectual fingerprint. Ideas that recur across years.

### [Cluster Name]
- [concept-slug](concept-slug.md) — one-line characterization
- ...

### [Other Cluster]
- ...

## Developing (T2) — N concepts
Sharpening. Might become canon.

## Speculative (T3) — N concepts
Testing in public.

## Stats
- Total concepts: N
- T1 Canon: N
- T2 Developing: N
- T3 Speculative: N
- T4 Riff: N
- Earliest source: YYYY-MM-DD
- Latest source: YYYY-MM-DD

Phase 5: Curation cull — keep/delete/merge rubric

Phases 1–4 only merge up — they never remove anything. Over months that leaves a corpus where hollow stubs dilute the concepts that actually compound. Phase 5 is the cull: a hard verdict per concept, run on a cadence or on demand, with every destructive step reversible.

Convention: see conventions/test-before-bulk.md — cull 3-5 clusters first, read the actual output, only then run the full pass.

The core question

If the user pulled this concept up cold in two years, would it sharpen a thought or seed something new — or would they scroll past it as filler?

Scroll-past = DELETE.

The 6 axes (score each 1-5)

Three substance axes weighted 2x, three packaging/fit axes weighted 1x. Substance carries the concept; packaging earns it surface area.

SUBSTANCE (2x weight):

PACKAGING & FIT (1x weight):

Scoring → verdict

Weighted score = (Insight + Originality + Specificity) × 2 + (Voltage + Representative + Powerful) × 1. Max = 45; express as %.

Minimum substance gate (overrides the %): a concept can NEVER be KEEP or ELITE if Insight < 3 or Originality < 3. Style does not buy its way past a hollow idea.

MERGE/REWRITE is a real third verdict, not a dodge. Many stubs have a live idea trapped in a weak body — fold those into the cluster canonical or rewrite them to stand alone. Use it when Insight ≥ 3 but Specificity or Voltage drags the score down.

Hard DELETE triggers (any one = delete, regardless of score)

• Fortune-cookie restatement — true but says nothing a greeting card wouldn't; platitude, no mechanism.
• Fragment — requires unavailable context; not self-contained (unless rare provenance, and even then only if intelligible + useful).
• Mangled extraction — transcription garble, truncated mid-thought, incoherent, or a chunk header masquerading as a concept.
• Off-mission trivia — accurate but unconnected to anything the user builds, believes, or could use.
• Duplicate within cluster — fails the operational duplicate test below.
• Unsupported factual claim — a factual/historical/causal assertion that's wrong or unsourced and stated as fact (see grounding labels). Soften-or-cut.

Grounding labels (factual concepts only) — label, don't just penalize

Any factual, historical, scientific, or causal claim gets a truth pass and a grounding: frontmatter label:

• VERIFIED — accurate + sourced → fine to keep and deploy.
• OPINION — clearly framed as the user's take or argument → fine.
• NEEDS_SOURCE — plausible but unsourced as-fact → keep only if reframed as claim/opinion.
• UNSAFE — wrong, or punchy-but-false → DELETE or soften.

Do not store confident falsehoods — deployed, they make the user less well understood, not more. Citations follow conventions/quality.md.

Reputational-risk gate

A concept that is punchy but could misrepresent the user — make them sound cruel, dismissive of people, or holding a position they don't — is a liability, not ammunition. Flag for rewrite or delete even if it scores high on voltage. Powerful means usable without blowback.

Cluster budget (the "trite at scale" problem)

When many concepts come from one source or share one idea, evaluate the SET, not each in isolation. Per semantic cluster, the default budget:

• 1 canonical concept (the sharpest statement of the mechanism) — always.
• +1-2 more ONLY if each adds a distinct mechanism, a concrete example, a different emotional register, a new audience, or singular phrasing from the user.
• More than 3 only if tied to an active project.

Everything else in the cluster is MERGE (preferred — see below) or DELETE. Forty near-identical stubs on one theme → one canonical mechanism concept, maybe one great line. The rest merge up.

Operational duplicate test

Don't eyeball "% overlap." Compare the candidate against the best existing concept in its cluster and ask: does this add a new mechanism, example, emotional register, audience, or user-specific phrasing? If no → MERGE (fold it in, keep the signal) or DELETE. If yes → the thing it adds is what justifies keeping it.

Hard KEEP overrides (rescue a low score — but floored)

Each override applies ONLY if the concept is intelligible and potentially useful:

• Singular voice — captures something only the user would say. Voice beats polish, but not voice over coherence.
• Load-bearing for an active project — directly feeds a known thesis or work in flight.
• Rare provenance — a real quote/moment that can't be regenerated (a meeting, the user's own note), AND it carries recoverable meaning. A content-free "great point about the AI thing" does NOT qualify.

Merge-with-backlinks (reversible — nothing is destroyed)

For redundant clusters the cull is INVERTED: do not delete the tail — merge it up into the canonical head and let the merge ledger become a salience metric. An idea independently re-derived N times isn't bloat; it's the corpus flagging this matters in N different contexts. Deleting dupes throws that signal away; merging captures it.

Each merge grows three frontmatter fields plus one body section on the canonical:

• merge_count (int) — raw number of pages absorbed, including same-source re-extractions.
• independent_sources (int) — distinct sources the cluster drew from. This is the true salience metric — raw merge_count inflates when one source gets re-extracted repeatedly; independent_sources is the fix.
• backlinks (list of {source, angle, date}) — every absorbed page's source plus the specific angle it brought. All framings survive; they just stop being separate top-level pages.
• ## Facets (body) — the canonical mechanism up top, then one short "as seen in {source}: {angle}" line per absorbed page. The concept becomes multi-angle, not redundant.

Merge-quality gate (reject incomplete merges): a merge is only written if (a) the ## Facets section has one line per absorbed page (source + specific angle) and (b) every backlinks entry has source + angle + date. Empty facets or dangling entries = reject the merge and flag the cluster for manual review. No half-merges.

Distinctness guard is a HARD VETO, not advisory. Two concepts that look like duplicates are NOT merged unless an LLM judge AFFIRMATIVELY confirms they state the SAME mechanism. Default is DON'T merge; the judge must earn the merge, and its yes/no + reason is logged per cluster. Different mechanisms/examples/registers → separate canonicals. Similarity proposes; judgment disposes.

Finding merge candidates — qualitative bands, not numeric cutoffs. Do not hardcode a similarity threshold: gbrain search returns hybrid (RRF-fused) scores, not raw cosine similarity, and any pinned number rots as the corpus and search mode shift. Work qualitatively: search each concept's title + first paragraph and treat another concept as a merge CANDIDATE when the two surface each other at the top of the result list with a visible score gap to the rest. Concepts that share vocabulary but not mechanism land mid-list — that's exactly the band where the distinctness guard earns its keep. Calibrate on your own corpus distribution before the bulk pass.