Schema Unify

Schema Unification (gbrain-base → gbrain-base-v2)

v0.41.22 ships gbrain-base-v2 — a 15-type DRY/MECE taxonomy (14 canonical + note catch-all) — as the install default for new brains. Existing brains on gbrain-base can opt in via the pack_upgrade_available onboard finding + the unify-types PROTECTED Minion handler.

This skill is the playbook for that migration.

brain_first: exempt

This skill is ABOUT the brain's shape — it can't depend on the brain it's reshaping. No gbrain search lookup first; jump straight to onboard.

When this skill fires

• Agent runs gbrain onboard --check and sees pack_upgrade_available or type_proliferation warnings
• User asks "what is the canonical taxonomy / how do I clean up my page types / migrate to v2"
• A dangling_aliases finding surfaces (post-unify GC)
• An agent ingesting from a custom pack wants to consult the v2 taxonomy as a reference

Mental model (one paragraph)

A production gbrain brain accreted 94 distinct pages.type values over years of ingestion: tweet / tweet-thread / tweet-bundle / tweet-single / media/x-tweet/bundle / tweet-stub all coexisting; 5.5K concept-redirect pages; atom-partner-link pages that should be links; civic / framework / insight / memo / anecdote one-offs. The cure: collapse to 15 canonical types (person, company, media, tweet, social-digest, analysis, atom, concept, source, deal, email, slack, writing, project, note) with subtypes/format/origin pushed to frontmatter, alias-rows for redirects, real link-rows for edge-shaped pages, and a catch-all that bins long-tail unknowns to note with frontmatter.legacy_type = <original> for rollback.

Workflow

Phase 1: Discovery

Confirm the brain is actually on gbrain-base (not already on v2).

gbrain schema active --json | jq -r '.identity'

Expected: gbrain-base@1.0.0+<sha>. If you see gbrain-base-v2@..., the brain is already on v2 — skip the migration.

Then run onboard to see what would change:

gbrain onboard --check

Look for the pack_upgrade_available finding. If it's ok, there's no successor declared for the active pack — done.

Phase 2: Preview

Run the per-cluster narrative:

gbrain onboard --check --explain

This invokes the unify-types handler in dry-run mode and prints:

• How many pages would retype per cluster (tweets, articles, companies, etc.)
• How many concept-redirect pages would become alias rows
• How many edge-shaped pages would convert to real links
• The synthesized catch-all rules for unknown types

Review the output. If the proposed changes look wrong, don't proceed — file an issue or write a custom pack with adjusted mapping_rules.

Phase 3: Apply

Managed brains: stop after the preview. Applying the migration is not supported on a managed brain (managed persistence on) yet: the retype writes pages directly instead of through the coordinated writer, so the apply job is refused (writer_coordinator_required) before it changes anything. The pack_upgrade_available finding says so on those brains. Keep using gbrain onboard --check --explain to preview, tell the user the pack upgrade waits for coordinated retype support, and don't submit the apply job below. One exception: when the preview shows nothing to retype, link or alias, the apply job only switches the active pack, so it may be submitted with the user's agreement.

The handler is PROTECTED (manual_only) — autopilot will never auto-fire it. Submit explicitly:

gbrain jobs submit unify-types \
  --params '{"target_pack":"gbrain-base-v2","apply":true}'

On PGLite (the install default), or on any setup without a running gbrain jobs work worker or supervisor daemon, add --follow so the job executes inline:

gbrain jobs submit unify-types \
  --follow \
  --params '{"target_pack":"gbrain-base-v2","apply":true}'

The persistent worker daemon is Postgres-only. Without --follow on PGLite, the job sits queued forever and the migration never runs.

apply defaults to false (dry-run) per the handler contract, so "apply":true is required here or the job reports success having retyped nothing and left the active pack unflipped. Omit it to preview.

Watch progress per phase (worker-daemon runs; with --follow the same progress streams inline):

gbrain jobs get <job_id>      # one job: status, progress, result
gbrain jobs watch --follow    # live dashboard of the whole queue

A job that stays queued here means no worker is running; resubmit with --follow to execute it inline.

On a 186K-page brain expect ~10 minutes. The handler runs:

1. Preflight (validate target pack has mapping_rules:)
2. Stats snapshot (pre-state for celebration summary)
3. Acquire gbrain-unify db-lock (60min TTL)
4. Apply phases:
   • Explicit retype rules (tweets, articles, companies, etc.)
   • Catch-all retype (unknown types → note with legacy_type)
   • Page-to-link rules (atom-partner-link, symlink)
   • Page-to-alias rules (concept-redirect)
5. Final sync (untyped rows by path-prefix)
6. Flip active pack to gbrain-base-v2
7. Verify + celebration summary

Phase 4: Verify

gbrain onboard --check
gbrain schema stats

Expected:

• pack_upgrade_available → ok (active pack is now v2)
• type_proliferation → ok (≤16 distinct typed values)
• dangling_aliases → ok (slug_aliases all point at active canonicals)
• gbrain schema stats shows ≤16 distinct types

Phase 5: Post-migration

Search and query --type article keep returning those pages post-unify: media declares article as an alias, so the type filter expands through the active pack's alias closure (the results also include other media pages). Direct SQL against pages.type needs updating to the canonical types.

Search queries get a small ranking signal: pages reached via slug_aliases (canonicals of one or more aliases) get a 1.05x boost. Visible via gbrain search --explain.

Rollback

Every retyped page preserves frontmatter.legacy_type = <original>.

Restore types in bulk (Postgres/Supabase deployments only; requires direct DB access):

UPDATE pages SET type = frontmatter->>'legacy_type'
WHERE source_id = 'default' AND frontmatter->>'legacy_type' IS NOT NULL;

On PGLite there is no SQL shell, so use the CLI surface instead: frontmatter.legacy_type persists per page, so individual retypes can be reverted through the normal put_page/CLI surface, and the soft-delete restore and pack-flip revert below work on every engine.

Page-to-alias and page-to-link source pages soft-delete with 72h TTL. Restore within that window:

gbrain restore <slug>

Revert the active pack flip:

gbrain schema use gbrain-base