Radar

/radar — Standing surveillance on the sources you care about

deep-research answers a question you asked. radar surfaces the answers to questions you haven't asked yet, from sources you already trust, every day, without you going to look.

Mental model

sources.yaml  →  fetch new only  →  score  →  digest  →  promote  →  raw/  →  /sb compile
  (config)        (state files)     (1–5)    (vault)    (you)      (vault)     (wiki)

Two hard rules keep this from becoming noise:

1. Only one thing is written per run by default — the digest. Individual raw/ captures happen only for items that clear the auto-capture bar, or that you explicitly promote.
2. Nothing is fetched twice. Every source has a state file of seen item IDs. A run at 7am and a manual run at 9am produce no duplicates.

Layout

<vault> is ${SECOND_BRAIN_VAULT:-$HOME/Documents/SecondBrain}.

Step 0 — Parse mode

If sources.yaml doesn't exist in any mode but add/schedule, run first-time setup: copy references/templates/sources.example.yaml into place, create state/, runs/, logs/, then walk the user through adding their first 3–5 sources. Don't run against the example file's placeholder sources.

Read references/sources-schema.md before touching sources.yaml in any mode.

Mode: run

Step 1 — Load

1. Read sources.yaml. Filter to enabled: true sources whose cadence is due (compare against each state file's last_run; daily = due if last run was on an earlier calendar day).
2. Read interests.local.md — this is the global relevance context every item gets scored against, on top of each source's own focus.
3. Read each due source's state file. Missing state file = first run for that source; use defaults.first_run_lookback_days (default 3) instead of "since last run" so a new source doesn't dump its entire archive.

Step 2 — Fetch, in parallel

Read references/fetchers.md for the exact command per source type. Fetch every due source in parallel — they're independent, and a serial run over 15 sources is the difference between a 40-second morning job and a 6-minute one.

Rules that matter more than they look:

• A failing source never fails the run. Catch per-source errors, mark the source degraded with the error text, and carry on. A dead RSS feed must not cost you the YouTube results.
• Cap per source at max_items_per_source (default 15). If a source blew past the cap, say so in the digest — it usually means the lookback is too wide or the source got noisy.
• Filter to new by ID against the state file's seen list, then by published against the lookback window. Both, not either: IDs catch re-publishes, dates catch feeds that recycle IDs.
• Every type has a dependable path now — the free ones (youtube / rss / hn / bluesky / mastodon / reddit) plus X and LinkedIn via ScrapeCreators at ~$0.002 a call. A source that still exhausts its chain is marked degraded and skipped; don't retry in a loop, don't let it block the digest.
• Filter X by created_at, never by position. ScrapeCreators returns pinned and high-engagement tweets interleaved with recent ones — a single verified call put a 2024 tweet second. Trusting the order makes radar "discover" years-old posts as new.
• Watch the credit balance. Every ScrapeCreators response carries credits_remaining; record it in the run record and warn in the digest below ~1,000.
• Check credentials once, at the start. Resolve AUTH_TOKEN/CT0 and any paid keys (env → OS keychain) before fetching, and skip the source types that need what's missing rather than discovering it per-item. references/fetchers.md → "Credentials" has the resolution order.

Step 3 — Score

For each new item, produce a relevance score 1–5 against the source's focus + interests.local.md:

Score from the title + description/excerpt + whatever the feed gave you. Do not fetch full content to score — that's backwards, and it's what makes daily jobs slow and expensive. Full fetch happens on capture only.

Then bucket:

• >= auto_capture_at (default 5) → capture now, in full (Step 4)
• >= list_at (default 3) → listed in the digest as promotable
• below list_at → collapsed into a "skipped" count with titles in a <details> block. Never silently dropped — a bad filter must be visible.

Write the score's reason in one clause. "Names the exact attribution problem TracerKit solves" is useful. "Relevant to your interests" is not, and if that's the best you can write, the score is a 3.

Step 4 — Capture the auto-captures

For each item at or above auto_capture_at, fetch the full thing and write it to <vault>/raw/ following second-brain's schema (read that skill's capture conventions; the vault's CLAUDE.md or AGENTS.md is authoritative):

Every captured file gets a header:

source: <url>
captured: YYYY-MM-DD
via: radar/<source-id>

The via: line is what makes a bad source auditable later — when the wiki fills with mediocre pages, you can trace which source produced them.

Step 5 — Write the digest

Render references/templates/digest.md to <vault>/outputs/radar/<YYYY-MM-DD>.md.

If a digest already exists for today (second run same day), merge: append the new items with continued numbering, update the run header's counts, and add a second run line. Never overwrite — the numbers in an existing digest may already have been used in a promote call.

Step 6 — Persist state

Per source, write state/<source-id>.json: last_run, last_status (ok | degraded | error), error if any, and seen — the item IDs, capped at the most recent 300 (a feed rarely revisits further back, and unbounded state files are how this rots).

Write runs/<YYYY-MM-DD>.json with the full item list, each with its digest number, score, URL, source ID, and capture status. promote reads this file, so it must contain everything needed to fetch an item without re-polling the source.

Step 7 — Commit the vault

Per the standing vault rule: git -C "<vault>" pull --rebase --autostash, commit the digest + captures in one semantic commit (radar: 2026-08-28 digest — 34 items, 3 captured), push. Pull first, always — an unattended job that force-diverges the vault is worse than one that doesn't run.

Step 8 — Report

In an interactive session, print the digest summary inline — run stats, the captured items, the top 5 promotable ones by score, and any degraded sources. Don't print the full skipped list.

In an unattended run (headless agent, e.g. claude -p or codex exec), print the same thing to stdout; the scheduler captures it to logs/<date>.log.

Mode: promote

/radar promote 3 7 12 — pull specific digest items into raw/ in full.

1. Read runs/<date>.json (today's by default; /radar promote --date 2026-08-26 4 for an older digest).
2. Resolve each number to its item. Refuse cleanly on a number that doesn't exist or was already captured — say which, don't guess at intent.
3. Fetch + capture each exactly as Step 4 does.
4. Update the run record (captured: true) and the digest note — move the promoted lines into the Captured section with their raw/ paths.
5. Commit + push.

Accept fuzzy references too: "promote the Isenberg one" → match against titles in the run record, confirm the match if there's more than one candidate.