Research With Sources

Research — gather sources and write provisional findings

This skill is pack guidance. The platform /open-knowledge skill (read/write/preview/linking/grounding rules) still governs every markdown operation — this layers the procedure on top.

Conduct evidence-driven research on a topic and produce a provisional research article under research/. Provisional, not canonical: research articles capture findings, trade-offs, and open questions at a point in time. They are promoted to canonical articles via the /consolidate-notes skill only when decisions solidify.

The content directory is the resolved content.dir — read it with config({ key: 'content.dir' }) if you don't already know it. Paths below are relative to it.

Three paths

• Path A — Research article (DEFAULT): A persistent provisional article with status: draft and an inline sources: frontmatter list pointing at raw sources captured via the ingest procedure. This is the default unless the user explicitly opts out.
• Path B — Direct answer: Findings delivered in conversation only. Requires explicit user request (e.g., "just tell me", "no doc needed", "quick answer").
• Path C — Update existing research: Surgical additions/corrections to an existing research article. Triggered when the user references an existing research doc or says "update/refresh/extend."

Path A is the default because provisional articles compound over time; spoken answers do not.

Legacy reads: Existing articles may use status: provisional and string paths under sources:. Treat those as draft research and source resources. Do not mass-rewrite them; new writes use the OKF shapes below.

Autonomy mode

In headless mode, propose the rubric AND proceed immediately. Mark the Scoping task completed after proposing.

Mandatory execution order

Hard gates — do NOT skip ahead. If you find yourself about to run a WebFetch or WebSearch without completing Steps 0-2, STOP — you skipped a gate.

1. Step 0: Create workflow checkpoint tasks — ALWAYS the first action.
2. Step 1: Scan existing coverage + route — scan the content directory for prior work; classify coverage; present options before new research begins.
3. Step 2: Collaborative scoping — propose a research rubric. In Supervised mode, STOP and WAIT for user confirmation before any external fetch.
4. Step 3: Capture raw sources via ingest — preserve before analyzing.
5. Step 4: Read + analyze — third-party/external by default; first-party codebase only when the user explicitly requests.
6. Step 5: Write the research article — Path A only.
7. Step 6: Link aggressively + file valuable Q&A back.
8. Step 7: Validate — frontmatter, dead-links, sources alignment.
9. Step 8: Recap + follow-up directions.

Path B shortcut: If the user explicitly requested a direct answer in Step 2, skip Steps 5 and 7. Steps 0, 1, 3, 4, 6, 8 still apply (evidence discipline doesn't relax just because output is conversational).

Report framing default: external / third-party sources

Research articles default to external framing — investigating third-party topics, technologies, concepts, public repos, papers, official docs. Do NOT mix the user's own codebase analysis into the research article unless the user explicitly asks. Mixing drifts findings from factual synthesis toward opinion-forming applied to the organization, reducing factual fidelity.

• Default: external sources (web, open-source repos, papers, official APIs).
• Exception: if the user asks "research how our X compares to Y" or "include our codebase," include it — but clearly separate first-party observations from third-party findings in the article so a reader can distinguish externally-verifiable facts from organization-specific takes.

Persist as you go — the article IS your checkpoint

PERSIST AS YOU GO — crash-safe checkpoint rule. The single most expensive failure this procedure produces is completed research lost to a mid-session rate limit or crash — analysis held in context, never written, discarded when the session died. The user paid for work that vanished. The knowledge base is the checkpoint; two rules make every step crash-safe:

• Ingest each source the moment you fetch it (Step 3), one at a time — never fetch all sources and ingest them in a trailing batch. An ingested source survives a crash; a fetched-but-unwritten one does not.
• Create the article skeleton early and fill it section-by-section as you read (Steps 4-5), not in one final write at the end. After you analyze each source, edit its findings into the article before moving to the next. A crash after reading five of eight sources then leaves five sections safely in the KB; you resume by reading the partial article back, not by re-running the whole sweep.

Structured notes that live only in your context are not persisted work. If a finding is worth keeping, it belongs in an ingested source or in the article — written, not held.

Step 0: Create workflow checkpoint tasks

ALWAYS THE FIRST ACTION. Before any read, any scan, any fetch — create tasks. They persist across context compaction, make skipped steps immediately visible, and show progress to the user.

Create these tasks via your host's task system (TaskCreate in Claude; equivalent elsewhere):

TaskCreate: "Research: Scan existing coverage + route"        → start as in_progress
TaskCreate: "Research: Collaborative scoping — rubric gate"   → pending, blocked by #1
TaskCreate: "Research: Capture sources via ingest"            → pending, blocked by #2
TaskCreate: "Research: Read + analyze"                        → pending, blocked by #3
TaskCreate: "Research: Write the research article"            → pending, blocked by #4
TaskCreate: "Research: Link aggressively + file Q&A back"     → pending, blocked by #5
TaskCreate: "Research: Validate (frontmatter + dead-links)"   → pending, blocked by #6
TaskCreate: "Research: Recap + follow-up directions"          → pending, blocked by #7

Use the host's blocked-by relation to enforce ordering. As you complete each step, mark the task completed and the next task in_progress.

Path B variant: If scoping determines Path B (direct answer), drop tasks #5 and #7 — they don't apply.

Path C variant: If Step 1 routes to Path C (update existing), drop tasks #3 and #5 (ingest is usually unnecessary and no new article is created) and rename task #4 to "Research: Read existing article + diff deltas."

Why tasks: the observed failure mode is the agent jumping straight to WebFetch without scanning or scoping. Tasks make the skipped gates obvious to the user mid-session.

Step 1: Scan existing coverage + route

MANDATORY FIRST RESEARCH STEP. Before any external fetch, scan what the knowledge base already holds.

Phase 1: Check existing knowledge

If the user explicitly references an existing research article (names it, links it, says "update/refresh/extend"): → Skip the scan. Go directly to Path C.

Otherwise, always scan first:

1. exec("grep -rln <topic-keyword> <content-dir>") — returns matching files with frontmatter enrichment so you can judge relevance without opening each.
2. exec("ls -A <content-dir>") — surfaces folder layout and most-recent-updated doc per subdir.
3. For the 1-3 most promising candidates, exec("cat <path>") — returns full doc + frontmatter + backlinks + recent shadow-repo activity.

Classify:

Phase 2: Present routing options (Supervised mode)

Fully covered:

"We already have research on this in <path>. Here's what it found: [2-4 key findings]. Options: (1) use as-is, (2) verify / refresh (article is from [date]), (3) extend on [specific dimension], (4) new angle if this is a different framing."

Let the user choose. Do NOT start new research when existing research already answers the question.

Partially covered:

"We have related research in <path> covering [scope]. Your question about [topic] isn't directly answered but it's a natural extension. Options: (1) extend existing via Path C, (2) start new article via Path A. I'd recommend [1 or 2] because [reason]."

Not covered:

Proceed to Step 2 (scoping). If the user asked for a quick answer, flag that Path B may apply and confirm in Step 2's scoping exchange.

Headless mode: auto-select — fully-covered → proceed to new article on the specific angle the caller requested; partially-covered → start new article; not-covered → Path A.

Scan discipline

• Do not skip the scan. Even 30 seconds of grep + cat prevents duplicate research AND gives the user context on what's already known.
• Bias toward extending (Path C) when topics are semantically coherent — one comprehensive article beats two overlapping ones.
• Bias toward new (Path A) when framing, audience, or primary question differs materially.

Step 2: Collaborative scoping (Supervised STOP gate)

HARD GATE (Supervised mode). Do NOT start external research until the user confirms the rubric. After proposing it, STOP and WAIT for user response. Only then mark the Scoping task completed.

In headless mode: propose the rubric AND proceed. Mark the task completed after proposing.

Propose a rubric

Return this structure to the user:

## Proposed research rubric

**Question:** [narrowed from the original topic — concrete, answerable, bounded]

**Dimensions to investigate:** [3-7 facets]
1. [Dimension 1]
2. [Dimension 2]
...

**Candidate sources:** [3-8 initial guesses]
- [Source 1 — why it's relevant]
- [Source 2 — why it's relevant]
...

**Success criteria:** [2-3 concrete outcomes — "the article cites X authoritative sources", "open questions are marked explicitly", etc.]

**Output format:** Path A (article) | Path B (direct answer) | Path C (update `<existing-article>`)

Scoping discipline

• If the original topic is vague ("research LLM agents"), narrow it before fetching: "What specific agents? For what decision? Over what time horizon?"
• If the topic is itself a URL, treat that URL as the anchor and widen to 2-4 adjacent authoritative sources.
• Name the decision this research informs. Research without a decision context meanders.
• Do not over-specify the rubric — the user can adjust. Propose, don't prescribe.

Step 3: Capture raw sources via ingest

For each relevant URL, paper, or document in the confirmed rubric, follow the ingest procedure — it lives in the platform /open-knowledge skill at references/ingest-and-sources.md, not in this pack. Typical research pulls 3-8 sources. Too few → thin synthesis. Too many → you'll be reading for the rest of the session.

• Don't skip ingest. Raw preservation separates capture from interpretation and makes research reproducible. An article without preserved sources is just opinion; an article with preserved sources is a trail someone else can follow.
• If a fetch fails for a source you specifically need, stop and ask the user to paste it — don't silently drop it. Write-time fabrication of missing evidence is the biggest failure mode.
• If a fetch returns an obvious summary instead of the raw bytes (some LLM-backed fetch tools do this), note it and try a raw alternative (curl -sL <url>, or ask the user to paste).