Hipocampus Compaction

Memory Compaction Tree

5-level hierarchical index over raw memory logs. Compaction nodes are search indices — originals are never deleted.

Hierarchy

memory/
  ROOT.md                      <- root node (topic index, ~3K tokens, Layer 1)
  2026-03-15.md                <- raw daily log (permanent, append-only)
  daily/2026-03-15.md          <- daily compaction node
  weekly/2026-W11.md           <- weekly compaction node
  monthly/2026-03.md           <- monthly compaction node

Compaction chain: Raw → Daily → Weekly → Monthly → Root

Tree traversal (search): Root → Monthly → Weekly → Daily → Raw

Fixed vs Tentative Nodes

Every compaction node has a status:

• tentative — period is still ongoing, regenerated when new data arrives
• fixed — period ended, never updated again

# Indicated in YAML frontmatter
---
type: weekly
status: tentative
period: 2026-W11
---

Key: tentative nodes are created immediately — ROOT.md is usable from day one.

When to Run

Called from hipocampus:core Session Start step 7, or directly by an external scheduler (e.g., OpenClaw heartbeat). Check trigger conditions below.

Trigger Conditions

Smart Thresholds

Below threshold: copy/concat verbatim (no information loss). Above threshold: generate LLM keyword-dense summary.

Type-Aware Compaction

Memory Types

Entries in daily logs are tagged: ## Topic [type]. Four types exist:

Backward compat: Untagged entries (no [type] in heading) → treat as [project].

Topics Keyword Extraction

At every compaction level, extract topic keywords from content and write to frontmatter topics field:

• Scan headings (## Topic [type]) for keywords
• Scan Key Decisions for decision keywords
• Include type tag: topics: [hipocampus [project], terse-responses [feedback]]

Exclusion Filtering

When generating LLM summaries, strip:

• Code blocks (triple backtick) → replace with → filepath:lines
• Stack traces → 1-line error message
• Entries containing "임시", "테스트 중", "나중에 삭제", "temporary", "test run", "delete later" → remove entirely

Algorithm

CRITICAL — STRICT CHAIN ORDER: Steps 2→3→4→5 MUST execute in sequence. NEVER skip a level.

Each step feeds the next. Root reads from monthly. Monthly reads from weekly. Weekly reads from daily. If you skip a level, the chain breaks and data is lost or corrupted.

Raw → [Step 2] → Daily → [Step 3] → Weekly → [Step 4] → Monthly → [Step 5] → Root
         ↑                    ↑                     ↑                    ↑
     reads raw           reads daily            reads weekly        reads monthly
     writes daily/       writes weekly/         writes monthly/     writes ROOT.md

NEVER:

• Modify ROOT.md based on daily or weekly data (root reads ONLY from monthly)
• Modify monthly based on daily data (monthly reads ONLY from weekly)
• Skip Step 2 or 3 because "there's nothing new" — always verify by checking files
• Touch ROOT.md directly without going through the full chain

Step 0: Pre-Compaction Snapshot

Before starting the compaction chain, preserve current working state:

1. Read WORKING.md
2. If it contains active task content (not just the empty template):
   • Append a snapshot to today's daily log (memory/YYYY-MM-DD.md):

     ## Working State Snapshot [project]
     - context: pre-compaction automatic snapshot
     - state: [copy WORKING.md content]

   • This ensures in-progress work is captured before any context compression
3. If WORKING.md is empty or contains only the template, skip this step

Step 1: Discover Candidates

Scan memory/ for raw files. Group by date, ISO week, and month. Check each group against trigger conditions.

Step 2: Daily Compaction (max 1 per cycle)

Input: raw files (memory/YYYY-MM-DD.md) Output: daily nodes (memory/daily/YYYY-MM-DD.md)

For each date where raw exists and daily needs create/update:

1. Read raw file memory/YYYY-MM-DD.md
2. Count lines — compare against ~200 line threshold
3. Below threshold: copy raw verbatim to memory/daily/YYYY-MM-DD.md
4. Above threshold: generate keyword-dense summary
5. Write with frontmatter:

---
type: daily
status: tentative
period: YYYY-MM-DD
source-files: [memory/YYYY-MM-DD.md]
topics: [keyword1, keyword2, keyword3]
---

## Topics
## Key Decisions
## Tasks Completed
## Lessons Learned
## Open Items

6. If date has changed (raw is from a past date): set status: fixed

Secret scanning: The mechanical compaction (hipocampus compact) automatically redacts secrets in compaction nodes using regex patterns. When generating LLM summaries for above-threshold nodes, also avoid reproducing any API keys, tokens, passwords, or credentials from the source material. If you encounter a secret in the source, write [REDACTED] in its place.

CHECKPOINT: Verify memory/daily/ has the updated file before proceeding to Step 3.

Step 3: Weekly Compaction (max 1 per cycle)

Input: daily nodes (memory/daily/YYYY-MM-DD.md) — NEVER raw files Output: weekly nodes (memory/weekly/YYYY-WNN.md)

STOP-CHECK: Did Step 2 produce or update a daily node? If not, skip Steps 3-5 entirely — there's nothing new to propagate.

For each ISO week where dailies exist and weekly needs create/update:

1. Read all daily compaction files for that week (from memory/daily/, NOT from memory/)
2. Count combined lines — compare against ~300 line threshold
3. Below threshold: concat all dailies
4. Above threshold: generate keyword-dense weekly summary
5. Write to memory/weekly/YYYY-WNN.md with frontmatter
6. If ISO week ended + 7 days elapsed: set status: fixed

CHECKPOINT: Verify memory/weekly/ has the updated file before proceeding to Step 4.

Step 4: Monthly Compaction (max 1 per cycle)

Input: weekly nodes (memory/weekly/YYYY-WNN.md) — NEVER daily or raw files Output: monthly nodes (memory/monthly/YYYY-MM.md)

STOP-CHECK: Did Step 3 produce or update a weekly node? If not, skip Steps 4-5 — there's nothing new to propagate.

For each month where weeklies exist and monthly needs create/update:

1. Read all weekly compaction files for that month (from memory/weekly/, NOT from memory/daily/)
2. Count combined lines — compare against ~500 line threshold
3. Below threshold: concat all weeklies
4. Above threshold: generate keyword-dense monthly summary
5. Write to memory/monthly/YYYY-MM.md with frontmatter
6. If month ended + 7 days elapsed: set status: fixed

CHECKPOINT: Verify memory/monthly/ has the updated file before proceeding to Step 5.