Pi History Ingest

Pi History Ingest — Session Mining

You are extracting knowledge from the user's Pi coding agent sessions and distilling it into the Obsidian wiki. Pi sessions are stored as structured JSONL with a tree layout — your job is to follow the active branch, extract durable knowledge, and compile it.

Session knowledge closure: Pi session files are the only factual source for this skill. Do not add background knowledge from model training, other tools, package docs, local files, or the current conversation unless that fact appears in the selected session entries. If outside context seems useful, mark it as an open question or skip it — never present it as extracted session knowledge.

This skill can be invoked directly or via the wiki-history-ingest router (/wiki-history-ingest pi).

Before You Start

Writing profile: Before drafting or rewriting natural-language Markdown, read and apply the Writing Profile Resolution section in llm-wiki/SKILL.md. Framework schema, provenance, safety, and operation-specific requirements take precedence. WRITING.md preferences apply only to newly drafted or rewritten natural-language Markdown; preserve source content and structured records.

1. Resolve config — follow the Config Resolution Protocol in llm-wiki/SKILL.md (inline @name override → walk up CWD for .env → global config → prompt setup). This gives OBSIDIAN_VAULT_PATH and PI_HISTORY_PATH (defaults to ~/.pi/agent/sessions)
2. Read .manifest.json at the vault root to check what has already been ingested
3. Read index.md at the vault root to understand what the wiki already contains

Ingest Modes

Append Mode (default)

Check .manifest.json for each source file. Only process:

• Files not in the manifest (new sessions)
• Files whose modification time is newer than ingested_at in the manifest

Use this mode for regular syncs.

Full Mode

Process everything regardless of manifest. Use after wiki-rebuild or if the user explicitly asks for a full re-ingest.

Pi Data Layout

Pi stores sessions under ~/.pi/agent/sessions/ (or the path set by PI_CODING_AGENT_SESSION_DIR).

~/.pi/agent/sessions/
├── --<cwd-path>--/                    # Working directory with / replaced by -
│   └── <timestamp>_<uuid>.jsonl       # Session JSONL file
└── ...

The session filename contains an ISO timestamp and UUID. The parent directory encodes the working directory where the session was created.

Session JSONL Format

Each .jsonl file is a sequence of JSON objects. The first line is always a session header; subsequent lines are tree entries with id and parentId.

Key entry types:

Message roles inside message entries

• user — user input; content is string or (TextContent \| ImageContent)[]
• assistant — assistant response; content is (TextContent \| ThinkingContent \| ToolCall)[]
• toolResult — tool execution result; content is (TextContent \| ImageContent)[]
• bashExecution — bash command + output; command, output, exitCode
• branchSummary — branch switch summary; summary string
• compactionSummary — compaction summary; summary string

Key data sources ranked by value

1. message entries (user + assistant) — full conversation transcripts; rich but noisy
2. compaction entries — pre-synthesized summaries of older context; gold
3. branch_summary entries — summaries of abandoned branches; good signal
4. bashExecution entries — concrete commands run; useful for workflow patterns
5. session_info entries — session name for topic inference

Skip model_change, thinking_level_change, custom (extension state), and label entries.

Step 1: Survey and Compute Delta

Scan PI_HISTORY_PATH and compare against .manifest.json:

# List all session files
find ~/.pi/agent/sessions -name "*.jsonl" -type f

# Or with custom path
find "$PI_HISTORY_PATH" -name "*.jsonl" -type f

Build an inventory. For each session file, record:

• path — absolute path
• cwd — decoded from parent directory name (--<path>-- → /path)
• session_name — from the latest session_info entry (if any)
• modified_at — file mtime
• already_ingested — presence in .manifest.json

Classify each file:

• New — not in manifest
• Modified — in manifest but file is newer than ingested_at
• Unchanged — already ingested and unchanged

Report a concise delta summary before deep parsing:

"Found N Pi sessions across K projects. Delta: X new, Y modified."

Step 2: Parse Session JSONL

For each selected session file, read it line by line. Because sessions use a tree structure, build the active branch first:

1. Parse all entries into a map by id
2. Find the current leaf (the entry with no children, or the last message entry)
3. Walk parentId chain from leaf to root to get the active path
4. Reverse the path so it's chronological

Extraction rules

From the active path, extract:

• session header — cwd, timestamp, parentSession (if forked)
• session_info — name field for session title/topic inference
• message entries with role: "user" — extract content text (skip images)
• message entries with role: "assistant" — extract text content blocks; skip thinking blocks (noise); note toolCall blocks (they reveal what the agent actually did)
• message entries with role: "toolResult" — summarize outcomes, not full output
• message entries with role: "bashExecution" — extract command + exit code; recurring commands reveal build/test/deploy workflows
• compaction entries — read summary verbatim; it's already distilled
• branch_summary entries — read summary verbatim; captures abandoned approaches

Evidence ledger

As you parse, build a private evidence ledger before writing any wiki page. Each durable fact or decision you may write must carry at least one source reference:

pi:<session-file-basename>#<entry-id>

If an entry lacks an id, use pi:<session-file-basename>:line<N> from the JSONL line number. Keep the cited text snippet or summarized observation next to the reference while drafting so you can verify claims before writing.

Skip / noise filters

• thinking content blocks — internal reasoning, not durable knowledge
• Image content blocks — skip unless the user explicitly asks for image transcription
• Raw tool outputs longer than 500 chars — summarize the outcome
• Token accounting (usage fields) — metadata only
• Repeated plan echoes or status updates

Critical privacy filter

Session logs can include injected instructions, tool payloads, and sensitive text. Do not ingest verbatim.

• Remove API keys, tokens, passwords, credentials
• Redact private identifiers unless relevant and user-approved
• Summarize bash outputs that contain paths, environment variables, or secrets
• Do not quote raw toolCall arguments verbatim if they contain sensitive data

Step 3: Cluster by Topic

Do not create one wiki page per session.

• Group knowledge by stable topic across many sessions
• Split mixed sessions into separate themes
• Merge recurring patterns across dates and projects only when each pattern member has evidence ledger references
• Use the cwd from the session header to infer project scope
• Use session_info.name as a topic hint when available
• Drop any cluster whose key claims cannot be traced back to the selected session files