Vellum Memory V2 Migration

Memory v2 Migration

Guided run for the first-time backfill of /workspace/memory/ from existing knowledge sources, ending with memory.v2.enabled = true and validated, embedded, ready-to-retrieve concept pages.

You are running memory consolidation — tending your personal wiki. The output is a cross-linked, cross-referenced collection of pages that is your memory, optimized for next-you. Care, judgment, voice. Your voice.

Procedure

⚠️ Do not run assistant memory v2 migrate during this skill. That command auto-generates concept pages from PKB and will overwrite any hand-written content without --force, and with --force will overwrite hand-written content silently. This skill replaces it with the hand-written path. If you've already started running this skill, treat migrate as off-limits until the migration is complete.

Step 0 — Read the principles

Read references/wiki-principles.md end-to-end before doing anything. It defines:

• Article shapes (event vs topic), gravity wells
• Class-by-folder taxonomy and size caps
• The cheat-sheet budget (10–20K tokens/turn) and fact density per byte
• Voice register by article shape
• Banned bullet shapes and the "one fact, one home" rule

The reference is the authoritative source for what a good page looks like. This SKILL.md owns what order to do things in.

Step 0.5 — Preflight

Three checks before dropping a sentinel commit. If any fails, stop and resolve before proceeding.

(1) CLI surface. Confirm the subcommands this skill calls are actually registered:

assistant --version
assistant memory v2 --help

The memory v2 help should list at least these four: validate, reembed, reembed-skills, activation. They're used in Steps 10 and 12. If any are missing, this skill assumes the post-cleanup CLI — either upgrade the binary or use the older migrate path on that workspace instead.

(2) Workspace state. If concepts/ is non-empty but partially populated (e.g. a previous run crashed mid-write), don't proceed under this skill — that's a recovery flow, not a fresh migration. Inspect with git log --grep memory-v2-migration to see how far the prior run got, then decide between resuming manually or rolling back to the last sentinel commit.

(3) Pin the migration to a high-quality model. Wiki backfill is judgment-heavy work — page routing, voice register, what-belongs-on-A-vs-B, when-to-stub-vs-not. A stronger model produces meaningfully better pages: better routing decisions, sharper bullet writing, fewer reflexive stubs. If your CLI exposes inference sessions, open one for the duration of the migration:

assistant inference session open quality-optimized --ttl 2h

If quality-optimized isn't a profile name on this workspace, list the available profiles and open the session against the one labelled highest-quality. Do not match on model name: the profile roster is per-workspace and the models behind each label change between releases.

assistant config get llm.profiles
assistant inference session open <profile-name> --ttl 2h

The --ttl 2h overrides the 30m default — comfortable headroom for a typical migration without leaving a forever-pinned session if the close in Step 14.5 is skipped. The session is conversation-scoped and stays active across all migration turns.

If assistant inference session isn't on your binary (older builds before the inference-session CLI shipped), proceed without it — the migration still works, the model just won't be pinned. Skip the close in Step 14.5 too.

(4) User confirmation. Before starting any work, confirm the user understands what they're signing up for. The migration is judgment-heavy LLM work — every concept page, buffer entry, and always-loaded file goes through inference. Duration and cost scale with the size of the existing knowledge base: a workspace with months of history and hundreds of buffer entries will take meaningfully longer and cost meaningfully more than a fresh one.

Use the CLI confirmation prompt:

assistant ui confirm "This migration will read all of your existing memories and knowledge base entries, distill them into concept pages, and re-embed everything. Depending on how long your assistant has been running and how many memories you have, this could take a while and cost real money. Proceed?"

If the user declines, stop the skill immediately — no sentinel commit, no work. If assistant ui confirm isn't available on this binary, ask the user directly in conversation instead.

Step 1 — Open a paper trail

/workspace is a git repo. Drop a sentinel commit so the migration is greppable in history:

cd /workspace
git add -A && git commit -m "memory-v2-migration: start" --allow-empty

You'll commit again at two more milestones (mid — after pages + buffer drain + always-loaded files, in Step 9; and at the very end, in Step 14). Three total sentinel commits, all using the exact prefix memory-v2-migration: so git log --grep finds them as one set. The heartbeat auto-committer may fire between your milestones — that's fine; the explicit sentinels are what make the migration story reconstructable later.

Step 2 — Inventory

Run in parallel:

ls /workspace/pkb/ 2>/dev/null
ls -R /workspace/memory/concepts/ 2>/dev/null
wc -c -l /workspace/memory/buffer.md /workspace/memory/essentials.md /workspace/memory/threads.md /workspace/memory/recent.md 2>/dev/null
assistant config get memory.v2.enabled

wc -l on buffer.md is the rough entry count; wc -c on the always-loaded files tells you how close they already are to budget.

Read every file in /workspace/pkb/ end-to-end. Read /workspace/memory/buffer.md end-to-end. Both are in scope — buffer is a parallel inbox of dated observations, not just PKB.

Step 3 — Plan

Following the reference's planning section, decide:

• Gravity wells — the 2–3 hub pages everything else edges to (the principal, the assistant, the org).
• Topic articles — what IS. Systems, tools, integrations, projects, named people, voice rules, places, recurring objects.
• Event articles — what HAPPENED. Landmark days, named launches, named conversations. Use sparingly; arcs are 10K-cap, topics are 5K-cap.
• Always-loaded files — essentials.md (static identity / org / standing rules, ≤10K), threads.md (active commitments, ≤10K), recent.md (time-windowed prose, ≤2K).

Step 4 — Default taxonomy (5 classes)

The original spec defaults to five class folders under memory/concepts/. Use these unless a specific need pushes elsewhere:

Sub-folders emerge as a class gets dense (people/colleagues/alice, objects/places/zurich-office). Don't pre-specify; let them emerge. Pages are cheap to move.

The slug is the relative path under concepts/ minus .md: alice, people/alice, procs/git-flow, arcs/2025-04-cutover.

Personalization is allowed but mixing is the bug. If you decide on a different layout (e.g. flat top-level system/, integration/, tool/ under concepts/), commit to it project-wide. Don't leave half the corpus under the default 5 and half under your custom layout — retrieval grows confused, edges break.

Step 5 — Article skeleton

Every page uses this shape:

---
edges:
  - path/to/sister
  - path/to/parent
ref_files:
  - pkb/source-file.md
summary: "1–5 sentence summary, ≤500 chars, plain prose only."
---
# title

- **bullet 1.** fact + implication folded in. inline pointer when bullet references another article → `path/to/article.md`.
- **bullet 2.** ...

Three path conventions in the same frontmatter — don't mix them up:

edges: route inside the wiki and participate in activation spread. ref_files: point outside the wiki to source material and are non-routable provenance pointers. Different roots on purpose.

Other format rules:

• summary: ≤500 chars, plain prose. No bullets, no bold, no italics, no emoji.
• 5–8 bullets per topic page. 10–12 per arc/event page.

Step 6 — Write the pages

Follow the reference's voice register and banned bullet shapes. "One fact, one home" is the foundational rule; the two below are tactics that flow from it:

• Trust adjacency. If page A edges to page B, and B holds fact X, do not restate X on A. Worked example: people/alice edges to objects/laptop. The laptop's brand, year, and dock setup live on objects/laptop. Alice's page just edges. Future-you searches "Alice's laptop" and gets both pages back via activation spread.
• Verify before encoding live status. For any in-flight work (open PRs, project state, integration health), verify against the actual repo / Linear / inbox before writing it down. Notes from prior sessions can be days stale, and the wiki is supposed to be ground truth, not a replay of stale notes.

Step 7 — Drain the buffer (if non-empty)

The buffer drain has two halves: the distillation (route facts onto concept pages) is judgment work and stays manual; the archival (move raw bullets to per-day files, reset buffer.md) is mechanical and should not be done by hand for 100+ entries.

For each dated bullet — judgment half:

1. Distill the long-lived fact onto the appropriate concept page (route, don't restate). If the bullet is purely transient ("worker restarted, log line spurious"), it can stay receipt-only — not every bullet needs a concept-page home.
2. Note inline if it contradicts existing wiki content (Step 6 rule: corrections land this pass).

For the archival half — use the helper:

python3 /workspace/skills/vellum-memory-v2-migration/references/buffer-drain.py --dry-run
python3 /workspace/skills/vellum-memory-v2-migration/references/buffer-drain.py

The helper is idempotent — re-running skips entries already present in the destination archive, so a partial-crash mid-drain is safe to recover from by re-running. It only resets buffer.md to a header-only file after a clean run with zero unparsed entries; unparsed entries are retained in buffer.md for human review rather than silently dropped.

If you'd rather inline a one-off snippet: Python 3 ships in the sandbox; the yaml module does not, so stick to the standard library. The helper is the reference shape.

Step 8 — Always-loaded files

Write or refresh:

• essentials.md (≤10K, target ≤4K): static identity facts about the principal, the assistant, the org structure, integrations status, standing rules. Reference register — terse and indexable.
• threads.md (≤10K): active commitments and in-flight work organized by status. Preserve any onboarding stubs from a pre-existing threads.md (avatar setup, memory imports, etc.) unless the user explicitly closes them — don't silently drop system-seeded items when rewriting the file.
• recent.md (≤2K): time-windowed prose, latest first, written in the assistant's voice.

If the shape of these files isn't already obvious from your context, see references/always-loaded-examples.md for fully-rendered ~30-line exemplars of each. Use them as shape guides, not content templates.