LLM Wiki

LLM Wiki — Knowledge Distillation Pattern

You are maintaining a persistent, compounding knowledge base. The wiki is not a chatbot — it is a compiled artifact where knowledge is distilled once and kept current, not re-derived on every query.

Three-Layer Architecture

Layer 1: Raw Sources (immutable)

The user's original documents — articles, papers, notes, PDFs, conversation logs, bookmarks, and images (screenshots, whiteboard photos, diagrams, slide captures). These are never modified by the system. They live wherever the user keeps them (configured via OBSIDIAN_SOURCES_DIR in .env). Images are first-class sources: the ingest skills read them via the Read tool's vision support and treat their interpreted content as inferred unless it's verbatim transcribed text. Image ingestion requires a vision-capable model — models without vision support should skip image sources and report which files were skipped.

Think of raw sources as the "source code" — authoritative but hard to query directly.

Don't confuse this with the in-vault _raw/ staging folder, which is a different thing: a scratch inbox for quick captures and drafts awaiting promotion (see wiki-capture and wiki-ingest). Files there aren't Layer 1 sources, but wiki-ingest still moves rather than deletes them on promotion, since some have no other copy.

Layer 2: The Wiki (LLM-maintained)

A collection of interconnected Obsidian-compatible markdown files organized by category. This is the compiled knowledge — synthesized, cross-referenced, and navigable. Each page has:

• YAML frontmatter (title, category, tags, sources, timestamps)
• Obsidian [[wikilinks]] connecting related concepts
• Clear provenance — every claim traces back to a source

The wiki lives at the path configured via OBSIDIAN_VAULT_PATH in .env.

Layer 3: The Schema (this skill + config)

The rules governing how the wiki is structured — categories, conventions, page templates, and operational workflows. The schema tells the LLM how to maintain the wiki.

Wiki Organization

The vault has two levels of structure: categories (what kind of knowledge) and projects (where the knowledge came from).

Categories

Organize pages into these default categories (customizable in .env):

Projects

Knowledge often belongs to a specific project. The projects/ directory mirrors this:

$OBSIDIAN_VAULT_PATH/
├── projects/
│   ├── my-project/
│   │   ├── my-project.md      ← project overview (named after project)
│   │   ├── concepts/          ← project-scoped category pages
│   │   ├── skills/
│   │   └── ...
│   ├── another-project/
│   │   └── ...
│   └── side-project/
│       └── ...
├── concepts/                   ← global (cross-project) knowledge
├── entities/
├── skills/
└── ...

When knowledge is project-specific (a debugging technique that only applies to one codebase, a project-specific architecture decision), put it under projects/<project-name>/<category>/.

When knowledge is general (a concept like "React Server Components", a person like "Andrej Karpathy", a widely applicable skill), put it in the global category directory.

Cross-referencing: Project pages should [[wikilink]] to global pages and vice versa. A project's overview page should link to the key concept, skill, and entity pages relevant to that project — whether they live under the project or globally.

Naming rule: The project overview file must be named <project-name>.md, not _project.md. Obsidian's graph view uses the filename as the node label — _project.md makes every project appear as _project in the graph, making it unreadable. So projects/my-project/my-project.md, projects/another-project/another-project.md, etc.

Each project directory has an overview page structured like this:

---
title: >-
    My Project
category: project
tags: [ai, web, backend]
source_path: ~/.claude/projects/-Users-name-Documents-projects-my-project
created: 2026-03-01T00:00:00Z
updated: 2026-04-06T00:00:00Z
---

# My Project

One-paragraph summary of what this project is.

## Key Concepts
- [[concepts/some-api]] — used for core functionality
- [[projects/my-project/concepts/main-architecture]] — project-specific architecture

## Related
- [[entities/some-service]] — deployment platform

Special Files

Every wiki has these files at its root:

Write them with obsidian-wiki memory, never by hand. index.md, log.md, hot.md, and the _meta/ tables share one advisory lock and are written atomically; hand edits in a parallel run drop whichever write lands second. obsidian-wiki memory sync <VERB> key=value does all three in one call. The full procedure — verbs, the Key Takeaways slot that stays yours, the owner profile and todo index — is in references/MEMORY.md.

index.md

A content-oriented catalog organized by category. Each entry has a one-line summary and tags. Rebuild this after every ingest operation. Format:

# Wiki Index

## Concepts
- [[transformer-architecture]] — The dominant architecture for sequence modeling ( #ml #architecture)
- [[attention-mechanism]] — Core building block of transformers ( #ml #fundamentals)

## Entities
- [[andrej-karpathy]] — AI researcher, educator, former Tesla AI director ( #person #ml)

Format rule: Add a space after the opening ( and tags. ❌ Don't: description (#tag) — breaks tag parsing ✅ Do: description ( #tag) — proper spacing and tag parsing

log.md

Chronological append-only record tracking every operation. Each entry is parseable:

## Log

- [2024-03-15T10:30:00Z] INGEST source="papers/attention.pdf" pages_updated=12 pages_created=3
- [2024-03-15T11:00:00Z] QUERY query="How do transformers handle long sequences?" result_pages=4
- [2024-03-16T09:00:00Z] LINT issues_found=2 orphans=1 contradictions=1
- [2024-03-17T10:00:00Z] ARCHIVE reason="rebuild" pages=87 destination="_archives/..."
- [2024-03-17T10:05:00Z] REBUILD archived_to="_archives/..." previous_pages=87

.manifest.json

Tracks every source file that has been ingested — path, timestamps, what wiki pages it produced. This is the backbone of the delta system. See the wiki-status skill for the full schema.

The manifest enables:

• Delta computation — what's new or modified since last ingest
• Append mode — only process the delta, not everything
• Audit — which source produced which wiki page
• Staleness detection — source changed but wiki page hasn't been updated

Source key contract (v2). Source keys — the sources keys in .manifest.json, the sources: frontmatter values on pages, and a project's source_repo — MUST be machine-portable. A vault is synced across machines, so a bare absolute path (/Users/..., /home/...) is never a valid stored key. This is the single canonical definition; other skills reference it rather than restating it.

Rules:

1. Never store a bare absolute path. Convert before writing, not after.
2. Normalize before comparing. Expand ~ and environment variables, resolve vault-relative keys against the vault root, and treat scheme:/:// pseudo-keys as opaque identifiers. Never compare raw strings without normalizing first.
3. Identity survives path changes. The same logical source keeps the same key across machines.
4. Pseudo-keys are an open namespace. What makes a key a pseudo-key is its shape (scheme: or ://, so it can never be mistaken for a file path), not a fixed list of names. Recommended names: repo:<host/owner/name> for a git project, url:<canonical-url> for a web page, agent:<agent>/<id> for an agent session. A source that is neither in the vault nor under $HOME still needs one — do not let it fall back to an absolute path.
5. Project identity is a repository, not a checkout. In the projects block, identify a project by source_repo (host/owner/name) rather than a machine path. A machine-specific checkout location, if useful at all, belongs in an optional source_cwd_hint (~-relative), never in the identity.

Reading is backward compatible: an existing manifest full of absolute keys keeps working, and scripts/manifest.py migrate <vault> --dry-run converts it to contract v2 (merging collisions, keeping the newest ingested_at). If the vault has moved between machines, its absolute keys are rooted at the old vault path, which matches neither the new vault nor $HOME — pass that old root explicitly with migrate <vault> --from-root <old-vault-root> (repeat the flag if the vault lived at more than one location). The command then reports nothing portable to write — N key(s) kept non-portable rather than claiming success. New writes go through the same normalization, so a skill may pass an absolute path to obsidian-wiki cache-update and still have a portable key land in the manifest.

Recording provenance. When you write a manifest entry, populate pages_created and pages_updated with the vault-relative page paths that source contributed to. This is what makes re-ingestion (when a source changes) able to find the pages to revisit, instead of guessing.

Page Template

When creating a new wiki page, use this structure:

---
title: >-
    Page Title
category: concepts
tags: [ml, architecture]
aliases: [alternate name]
relationships:
  - target: "[[concepts/related-concept]]"
    type: extends
sources: [papers/attention.pdf]
summary: >-
    One or two sentences, ≤200 chars, so a reader (or another skill) can preview this page without opening it.
provenance:
  extracted: 0.72
  inferred: 0.25
  ambiguous: 0.03
base_confidence: 0.65
lifecycle: draft
lifecycle_changed: 2024-03-15
tier: supporting
created: 2024-03-15T10:30:00Z
updated: 2024-03-15T10:30:00Z
# Optional. Written only by `obsidian-wiki snapshots set` / `apply` — do not hand-author.
# Quoted wikilinks with |title: Obsidian Properties does not treat Markdown [text](path) as links.
snapshots:
  - "[[_raw/_archived/example-clip|example-clip]]"
---

# Page Title

One-paragraph summary of what this page covers.

## Key Ideas

- The source's central claim, paraphrased directly.
- A generalization the source implies but doesn't state outright. ^[inferred]
- A figure two sources disagree on. ^[ambiguous]

Use [[wikilinks]] to connect to related pages.

## Open Questions

Things that are unresolved or need more sources.

## Sources

- [[_raw/_archived/example-clip.md]] — snapshot this page was distilled from

Sources section (required, last body section). Every wiki page ends with ## Sources. Entries must be clickable in Obsidian:

• Local snapshot (raw ingest, dropped PDFs/images, Web Clipper files, anything that landed in _raw/ and was archived): [[_raw/_archived/<filename>]] in the body Sources section (body wikilinks may include .md). YAML sources: stays origin keys (url:, agent:, repo paths, …), not the archive path. YAML snapshots: is separate: after moving a file to _raw/_archived/, run obsidian-wiki snapshots set <page> --archive _raw/_archived/<filename> then obsidian-wiki cache-update on that archived path. The CLI writes a List of quoted wikilinks with display text, e.g. "[[_raw/_archived/clip|clip]]" (no .md in the target; |clip is what Properties shows). Do not put Markdown [title](path) in snapshots: — Properties leaves those as unclickable text. The snapshots CLI does not touch the body. Do not link the webpage recorded in clipping frontmatter — that URL is mutable origin metadata.
• Fetched URL (/ingest-url with no saved snapshot): a markdown link to the canonical URL, and YAML sources: as url:<canonical-url>.
• Do not mix those up. A clip of a page is not an ingest-from-URL.

Related wiki pages stay in Related / relationships:, not in Sources.

Parser-safe scalars. Write free-text frontmatter values — at minimum title and summary — with folded scalar syntax (>-) as shown above: a bare scalar containing : (colon + space), #, or quotes breaks YAML parsing, and Obsidian then reports "Invalid properties" and hides the frontmatter. Keep the value indented on the line(s) following title: >- / summary: >-.

Paper Deep-Dive Template

The generic template suits most sources. Academic papers are the exception. For ML/AI/LLM/VLM (and similar) papers landing in references/, the substance lives in the architecture, the equations, and the results table — exactly what a terse "Key Ideas" list flattens away. For these, use the richer template below. This is the one place where "compile, don't retrieve" yields to a thorough, self-contained walkthrough a reader could study instead of the paper.

Obsidian renders the needed primitives natively, so no extra tooling is required: Mermaid fenced diagrams, $$…$$ LaTeX (MathJax), markdown tables, and ![[image]] / ![[paper.pdf#page=N]] embeds.

Use this template only when the source is an academic paper (arXiv/conference) with load-bearing figures or equations. Everything else uses the generic Page Template above. Frontmatter, provenance markers, confidence, lifecycle, and relationships: are unchanged — only the body sections differ.

---
# ...required frontmatter, same as the generic template; category: references...
---

# Paper Title

> [!tldr] One sentence: what's new, plus the headline result.

## Problem & Motivation

What's broken or missing that this paper addresses.

## Method / Architecture

Prose walkthrough. Embed the paper's real architecture figure as the primary
visual (see *Academic papers* in `wiki-ingest` for the PyMuPDF extraction recipe).
Fall back to a Mermaid flowchart only when no figure can be extracted.

![[attachments/<slug>-fig1.png]]
*Figure N (Author Year): one-line caption.*

## Key Equations

The 1–3 core equations as display math, not backtick code:

$$ \mathcal{L} = \mathbb{E}_{x}\!\left[-\log p_\theta(y \mid z)\right] $$

## Results

Headline numbers as a table, not a comma-separated blob — and embed a key
results/motivating figure (scaling plot, benchmark chart, capability collage)
when the paper has one:

| Method | Benchmark | Metric | Cost |
|---|---|---|---|
| Baseline | … | … | … |
| **This paper** | … | … | … |

![[attachments/<slug>-resultsN.png]]
*Figure N (Author Year): one-line caption.*

## Limitations

What the paper concedes or sidesteps. Mark reading-between-the-lines as ^[inferred].

## Related

Typed `[[wikilinks]]` to neighbouring work.

## Sources

- [[_raw/_archived/paper.pdf]] — snapshot distilled (if a local PDF/clip was ingested)
- <https://arxiv.org/abs/XXXX.XXXXX> — only if this ingest fetched the URL and there is no local snapshot

A Mermaid diagram reconstructed from the paper's prose is a synthesis, not a transcription — treat it as ^[inferred] when the interpretation is non-trivial.

Provenance Markers

Every claim on a wiki page has one of three provenance states. Mark them inline so the reader (and future ingest passes) can tell signal from synthesis.

These are framework defaults. A vault's AGENTS.md may add markers or workflow flags. Preserve owner extensions and treat orthogonal workflow flags separately from the extracted/inferred/ambiguous truth-state axis.

Example:

- Transformers parallelize across positions, unlike RNNs.
- This is why they scale better on modern hardware. ^[inferred]
- GPT-4 was trained on roughly 13T tokens. ^[ambiguous]

Why this syntax:

• ^[...] is footnote-adjacent in Obsidian — renders cleanly and never collides with [[wikilinks]].
• Inline (suffix) so a single bullet stays a single bullet.
• Default = extracted means existing pages without markers stay valid.

Frontmatter summary: Optionally surface the rough mix at the page level so the user can scan for speculation-heavy pages without reading them:

provenance:
  extracted: 0.72   # rough fraction of sentences/bullets with no marker
  inferred: 0.25
  ambiguous: 0.03

These are best-effort numbers written by the ingest skill at create/update time. wiki-lint recomputes them and flags drift. The block is optional — pages without it are treated as fully extracted by convention.

Typed Relationships

Plain [[wikilinks]] in page bodies carry no semantic weight — they indicate "related to" but not how. The optional relationships: frontmatter block adds typed, directional edges to the knowledge graph.

The relationships: block

relationships:
  - target: "[[Transformer Architecture]]"
    type: extends
  - target: "[[LSTM]]"
    type: contradicts
  - target: "[[Attention Mechanism]]"
    type: implements

Each entry has two required fields:

• target — a wikilink (using the same format as OBSIDIAN_LINK_FORMAT) to the related page
• type — one of the allowed semantic types below

Allowed relationship types

The table below is the framework default allowlist. A vault's AGENTS.md may extend it; consumers must use the effective allowlist and preserve owner semantics without coercion.

Rules

• Optional field — omit the block entirely if no typed relationships are known. Untagged wikilinks remain valid and are treated as related_to by wiki-export.
• Don't duplicate — if [[foo]] already appears as an inline wikilink, the relationships: entry just enriches it with a type; it is not a second link.
• Direction matters — the page declaring the entry is the source; target is the destination. Only declare relationships from this page's perspective.
• Don't fabricate — only add a typed entry when the source material makes the relationship direction and type clear. When in doubt, use related_to or omit.

Skills that read relationships:: wiki-export (emits typed edges), cross-linker (writes typed entries when inferring links), wiki-query (surfaces type in answers and walks the typed-edge graph for multi-hop "how is X connected to Y" path queries — bounded BFS over the relationships: adjacency, frontmatter-only).

Confidence and Lifecycle

Every page carries two orthogonal trust signals plus an optional supersession link.

The requiredness and lifecycle values below are framework defaults. A vault's AGENTS.md may extend lifecycle values or make trust fields optional. Validators must apply that effective owner schema while still validating any trust value that is present.

The deterministic lint/trust consumer accepts owner schema through OBSIDIAN_ALLOWED_LIFECYCLES, OBSIDIAN_ALLOWED_RELATIONSHIP_TYPES, OBSIDIAN_REQUIRED_TRUST_FIELDS, and OBSIDIAN_SCHEMA_SOURCE. Resolution precedence is CLI > environment/config > these framework defaults (with lifecycle and relationship extensions additive). Explicit blank or whitespace-only values fail closed; omit the variable to select defaults. wiki-lint/SKILL.md owns the operational invocation contract.

Required fields

base_confidence: 0.65          # [0.0, 1.0] — time-independent quality estimate. Stored once, recomputed on content change.
lifecycle: draft               # draft | reviewed | verified | disputed | archived
lifecycle_changed: 2024-03-15  # ISO date of last state transition
# lifecycle_reason: "..."      # optional free-text — why the state changed; surfaced by wiki-query
# superseded_by: "[[new-page]]" # wikilink; only when lifecycle=archived

lifecycle_reason and superseded_by are optional. Never fabricate them.

Confidence formula

The formula is a manual base score, not a deterministic URL classifier:

base_confidence = lineage_count_score * 0.5 + source_quality_score * 0.5

lineage_count_score  = min(independent_evidence_lineages / 3, 1.0)
source_quality_score = avg(reviewed quality score per independent lineage)

After calculating the raw score, assess whether the evidence covers the page's material claims. Partial coverage may justify keeping or lowering the score; unsupported material claims require source/claim repair before any confidence change. Avoid small score churn without meaningful epistemic change.

Source-quality scores (use the highest-matching bucket):

An independent evidence lineage is an origin that can corroborate a claim independently. Canonical source IDs remain useful for identity, but identity alone does not prove independence. Collapse dependent evidence before counting:

• files, releases, and commits from one repository → one repository lineage;
• retry/review/fix tasks in one workstream → one task lineage;
• parent/child Kanban records → one task lineage;
• byte-identical memories across profiles → one memory lineage;
• a snapshot plus the mutable source it captures → one lineage;
• aliases or metadata references resolving to one origin → one lineage.

The deterministic wiki-lint path validates _meta/trust-ledger.json; it does not recompute confidence from source strings. New or materially changed pages are marked for manual review. Refresh the ledger only after explicit human approval.

Per-skill defaults (ingest skills compute this automatically):

Lifecycle state machine

Five states. stale is not a state — it is a computed overlay: is_stale = (today − updated) > 90 days.

Only ingest skills set draft. All other transitions require a human editor. Update lifecycle_changed whenever the state changes.

Two edge classes are therefore illegal and are reported by obsidian-wiki lint as illegal_lifecycle_transitions: anything falling back to draft (reviewed|verified|disputed → draft), and any exit from archived (it is terminal — restoring a page is a deliberate delete-and-recreate, not a transition). The check compares against the lifecycle recorded in _meta/trust-ledger.json at the page's last review, so it only sees pages that have been reviewed at least once.

Importance Tiering

The tier: field controls which pages get updated on each ingest pass and their priority in retrieval. As wikis grow, re-reading every page on every ingest wastes tokens — tiering lets ingest and query skills focus effort where it matters most.

Three tiers