Paper Graph

Paper Graph

Build a Markdown report with embedded Mermaid diagrams showing how research on a user-specified topic (or paper) evolved — clustered into challenges → solutions and traced as per-solution evolution paths.

The skill has no outbound LLM dependency. The host agent provides all LLM calls; the skill provides deterministic data fetchers (S2 / DeepXiv), prompt templates, markdown parsers, and Mermaid renderers. Run the runbook below step-by-step.

Execution requirement. When the caller already provides parsed_query.json, seed.json and papers.json (wherever they are, for example in an input/ directory), copy them into the workdir under those names, treat steps 1–5 as complete and start at step 6. Still run every deterministic CLI step from classify through assemble; do not replace parsing, edge audit, rendering, or assembly with hand-authored Mermaid. Before finishing, verify that every parsed solution has a detail render and an audit verdict file, even when that file is an empty JSON list. Treat assemble_report as the only final-report writer: do not edit or append hand-authored edges afterward, even when a caller asks to display uncertain relationships. Preserve uncertainty in verdict JSON.

When to Use This Skill

Trigger when the user asks something like:

• "Show me the history of <topic>" / "How did <topic> evolve?"
• "Where does <paper> stem from?" / "What did <paper> build on?"
• "What are significant improvements / follow-ups to <paper>?"
• "Trace the lineage of ideas in <field>" / "Give me a literature taxonomy of <field>"
• "Citation tree of <paper>" / "Idea trace of <topic>"

Skip when:

• The user just wants a one-paper summary or single search hit (no relational/evolutionary aspect).
• The request is for non-academic citation work.
• The user explicitly wants a plain bibliography rather than a graph.

Inputs and Output

Inputs:

• A research query: a topic, a seed paper title/citation, or a hybrid. Free-form text.
• Output path (required): path for the final Markdown report. If not given, ask before running.
• (Optional) number of papers to fetch (--n flag on fetch_papers). Default 10.
• (Optional) Mermaid theme light or dark (--theme on render steps, or MERMAID_THEME env). Default light.

Output: a single Markdown file at the user-specified path with these sections:

1. Research goal (extracted from the query)
2. High-level taxonomy — one Mermaid graph: root → challenges → solutions → paper references
3. Per-solution evolution paths — one Mermaid graph per solution, showing paper-to-paper "evolution from" edges, evolution points, open challenges
4. Paper appendix — the numbered list of papers with title / year / authors / abstract / conclusion excerpt

Mermaid is just text inside ```mermaid fences — the file renders directly in GitHub, Obsidian, VS Code with the Mermaid extension, etc.

Setup

Required env (the skill fails verbosely if missing):

• S2_API_KEY — Semantic Scholar API key.

Optional env:

• DEEPXIV_API_TOKEN (or DEEPXIV_TOKEN) — DeepXiv arXiv search fallback. Install with pip install deepxiv-sdk; auto-provision with deepxiv token. Without it, S2 must fill the cite-number budget on its own.
• MERMAID_THEME — light (default) or dark. Overridden by --theme on each render call.

LLM: the host agent uses its own model and API key. The skill emits prompt templates and parses responses; it does not authenticate or call any LLM provider.

Working directory: create one dir per run, anchored relative to your current working directory (e.g. ./<basename>.work/). Avoid absolute system paths like /tmp/... — some harnesses sandbox the shell and the file-read tools to different filesystem roots, so an absolute path can appear writable to one and missing to the other. A cwd-relative path works the same everywhere. Run pwd once at the start if unsure.

Suggested layout (assuming final report goes to <output>):

<output>.work/
├── query.txt                user query (verbatim)
├── seed.json                resolve_seed_papers
├── seed_block.txt           format_seed_block
├── parsed_query.json        LLM: parse_query output
├── goal_block.txt           build_goal_block (reused in steps 8, 10, 11)
├── papers.json              fetch_papers → prefetch_sections → classify-merged
├── papers_input.txt         format_papers (full set)
├── classify_raw.json        LLM: classify output (consumed by merge_classifications)
├── core_filter.json         compute_core_filter (+ core_filter.json.allowed.txt)
├── papers_input_core.txt    format_papers (CORE-only; + papers_input_core.txt.allowed.txt)
├── outline_raw.md           LLM: outline output
├── outline.json             parse_outline summary
├── outline_mermaid.json     render_outline_mermaid
├── solutions/<key>.json     per-solution context (one file per solution)
├── parsed/<key>.json        parse_detail output (used by step 11 audit)
├── details/
│   ├── <key>_input.txt      format_papers (optional: same content as papers_input_core.txt; + .allowed.txt sibling)
│   ├── <key>_raw.md         LLM: detail output
│   └── <key>.json           render_detail_mermaid (consumed by assemble)
├── verdicts/<key>.json      [{source_n, target_n, verdict, source_quote, target_quote, reason}] from audit ([] when no edges)
└── <run>.log.jsonl files    one next to each subcommand output, default-on

Filenames are agent-chosen — these are recommendations to match what the runbook below references. <key> is the solution key string <s_major>.<s_minor> (e.g. 1.1). Every format_papers call writes a sibling <out>.allowed.txt containing the (N), (N), ... form ready to drop into a {allowed_numbers} placeholder.

Runbook

Each step is either a CLI call (python scripts/cli.py <subcmd> …) or an LLM call (the host agent reads a template from references/, substitutes the placeholders, and calls its model). All script and reference paths are relative to this skill's directory. CLI dependencies: pip install deepxiv-sdk httpx python-dotenv (also listed in requirements.txt at the skill root) — install into the environment the user is working in. Run sequentially; the detail step (step 10) and the audit step (step 11) can fan out per-solution / per-edge if the host supports concurrent tool calls.

Placeholder syntax in references/*.md. Single-brace {name} is a template slot the host must substitute. Double-brace {{...}} is an escaped literal brace — it appears in the prompt body when the example JSON the LLM is asked to emit contains braces. Substitute only single-brace slots; leave {{ and }} alone (they're for the LLM to read as { and } in its output).

Step 1 — Save the user query

Write the user's verbatim query to <workdir>/query.txt. Do not paraphrase. Multi-line queries are fine.

Step 2 — resolve_seed_papers (CLI, deterministic)

python scripts/cli.py resolve_seed_papers \
    --query-file <workdir>/query.txt \
    --out <workdir>/seed.json

Outputs a JSON array of S2-shape paper records for any arxiv IDs detected in the query. Empty array when none are present or all lookups fail.

Step 3 — format_seed_block (CLI, deterministic)

python scripts/cli.py format_seed_block \
    --seed <workdir>/seed.json \
    --out <workdir>/seed_block.txt

Renders the seed papers into the {seed_block} prompt fragment used in step 4. Empty input → empty output (the placeholder collapses cleanly).

Step 4 — parse_query (LLM)

Read references/parse_query.md. Substitute {seed_block} with the contents of <workdir>/seed_block.txt and {query} with the contents of <workdir>/query.txt. Call the LLM (low temperature, ~0.1).

Parse the response as JSON with this exact shape:

{"goal": "<one sentence>",
 "searches": ["<keyword phrase 1>", "<keyword phrase 2>", "..."],
 "definitions": {"<term>": "<one-sentence definition>", "...": "..."}}

Per the prompt's own instructions, searches should contain 2–4 phrases each of 2–5 keywords (these are two different counts — phrases vs. keywords-per-phrase).

Strip code fences if the model added them. Validate that goal is non-empty and searches is a non-empty list of strings. If the response is malformed, re-prompt the LLM once; on second failure, abort with a clear error message.

Save the validated JSON to <workdir>/parsed_query.json.

Step 5 — fetch_papers (CLI, deterministic)

python scripts/cli.py fetch_papers \
    --parsed-query <workdir>/parsed_query.json \
    --seed <workdir>/seed.json \
    --n 10 \
    --out <workdir>/papers.json

Uses S2 multi-search (one S2 query per searches[] entry) with DeepXiv fallback to top up to --n papers. Output is a JSON array of S2-shape paper dicts. If S2 returns nothing and DeepXiv is unavailable, exits non-zero — re-prompt step 4 with sharper search phrases.

Step 6 — classify (LLM)

First build the full {papers_input} block:

python scripts/cli.py format_papers \
    --papers <workdir>/papers.json \
    --out <workdir>/papers_input.txt

Read references/classify.md. Substitute {goal} with the goal_sentence (the single-sentence string at parsed_query.json["goal"] — bare, no definitions block here) and {papers_input} with the text file above. Call the LLM (low temperature, no reasoning needed — it's a discrete cataloging decision). Strip any leading/trailing code fences if the model added them.

Save the raw response verbatim to <workdir>/classify_raw.json. Expected shape:

{"classifications": [
  {"n": 1, "label": "CORE", "reason": "..."},
  {"n": 2, "label": "ADJACENT", "reason": "..."},
  {"n": 3, "label": "REJECT", "reason": "..."},
  ...
]}

Then merge into papers.json via the CLI (validates shape + applies failure-soft fallback to every-CORE if validation fails):

python scripts/cli.py merge_classifications \
    --classifications <workdir>/classify_raw.json \
    --papers <workdir>/papers.json \
    --out <workdir>/papers.json

merge_classifications always exits 0; on validation failure it applies an all-CORE safety-net and prints a … (FALLBACK: <reason>) suffix on the stdout success line. When you see that suffix, re-prompt the LLM once; if the second attempt also falls back, accept the all-CORE result and proceed — every paper just feeds the outline as CORE. On a clean run the success line shows the per-label counts and no FALLBACK suffix. Downstream subcommands (parse_outline, the renderers, assemble_report) read these labels via _label_of.

Step 7 — prefetch_sections (CLI, deterministic, best-effort)

python scripts/cli.py prefetch_sections \
    --in <workdir>/papers.json \
    --out <workdir>/papers.json

Mutates each paper dict in place by setting _conclusion_section. REJECT-labeled papers are skipped to save quota. Failures are silent; a paper without an arxiv ID or without a fetchable section just gets _conclusion_section: null.

After this step, any subsequent format_papers call automatically embeds each paper's _conclusion_section into the formatted block (under a Discussion/Conclusion excerpt: header) — that's how the outline and detail prompts get the OC-source signal the prompt templates reference. No extra step required.

Step 8 — outline (LLM)

Materialize the two prompt fragments that recur from here on — goal_block (the {goal} substitution) and core_filter (the CORE-only filter + its {allowed_numbers} sidecar) — as files on disk. Steps 10 and 11 read these files back, so the agent never has to carry them as conversation state.

python scripts/cli.py build_goal_block \
    --parsed-query <workdir>/parsed_query.json \
    --out <workdir>/goal_block.txt

python scripts/cli.py compute_core_filter \
    --papers <workdir>/papers.json \
    --out <workdir>/core_filter.json

compute_core_filter writes the index JSON to --out and the matching (N), (N), ... {allowed_numbers} form to a sibling <out>.allowed.txt. A paper without a classification counts as CORE, as everywhere else in this skill. On a no-CORE classifier outcome it falls back to every paper and prints (FALLBACK: …) — the run continues.

Build the CORE-only {papers_input} (also emits a sibling .allowed.txt, redundant here but consistent with Step 10):

python scripts/cli.py format_papers \
    --papers <workdir>/papers.json \
    --filter <workdir>/core_filter.json \
    --out <workdir>/papers_input_core.txt

Read references/outline.md. Substitute:

• {goal} — contents of <workdir>/goal_block.txt.
• {papers_input} — contents of <workdir>/papers_input_core.txt.
• {allowed_numbers} — contents of <workdir>/core_filter.json.allowed.txt.

Call the LLM (low temperature, ~0.2; allow ~8000 max tokens). Strip any leading/trailing code fences from the response — the prompt instructs the model to emit raw Markdown but a fence sometimes slips through. Save the cleaned Markdown to <workdir>/outline_raw.md.

Step 9 — parse_outline (CLI, deterministic)

python scripts/cli.py parse_outline \
    --raw <workdir>/outline_raw.md \
    --papers <workdir>/papers.json \
    --out <workdir>/outline.json \
    --solutions-dir <workdir>/solutions

Writes the outline summary plus one solutions/<key>.json context file per solution. Schema of each context file (consumed in steps 10 and 12):

{
  "challenge_idx": 1,
  "challenge_name": "Quality of random-projection targets",
  "solution_key": [1, 1],
  "solution_key_str": "1.1",
  "solution_name": "Optimization of the BEST-RQ pre-training objective",
  "paper_nums": [1, 3],
  "allowed": [1, 2, 3, 4]
}

paper_nums is the valid, de-duplicated primary taxonomy membership; when the outline repeats a paper, its first placement wins. allowed contains all CORE papers so detail generation can recover a canonical predecessor or successor without duplicating its primary taxonomy placement. format_papers --filter <solutions/key.json> and parse_detail --context <solutions/key.json> both read allowed automatically. Use paper_nums only for the {primary_numbers} prompt field.

A solution header that the outline restates reopens the same solution. A solution left without any valid paper (all its numbers were hallucinated or already placed elsewhere) is pruned: it gets no context file and no node in the taxonomy, and the success line on stdout names it. So every context file has a non-empty paper_nums.

Exits with code 4 and a stderr diagnostic if the outline contained no parseable ## Challenge N: headers, or if no solution is left with a valid paper. Re-prompt step 8 once; on second failure, lower --n in step 5 or sharpen the query.

Step 10 — detail (LLM, per solution)

CRITICAL — read "Fan-out task brief" below BEFORE delegating this step to subagents. A subagent prompt that points at SKILL.md or asks to "run paper-graph for solution X" will restart the whole workflow from Step 1 in a separate workdir, and its output will be unusable by the parent run.

Read references/detail.md once at the start of this step; substitute per-solution in the orchestrator. The subagents you fan out to never read templates themselves.

For each <workdir>/solutions/<key>.json produced in step 9:

(a) The per-solution {papers_input} is the CORE pool: every solution context's allowed is the full CORE set, so the block is the same for every solution and identical to <workdir>/papers_input_core.txt from Step 8 (with its .allowed.txt sibling). Reuse those two files. Running format_papers --filter <workdir>/solutions/<key>.json --out <workdir>/details/<key>_input.txt per solution still works and writes the same content; it is only needed if Step 8's files are gone.

(b) Substitute into references/detail.md:

• {goal} — contents of <workdir>/goal_block.txt (the file built in Step 8).
• {challenge_name} — from the solution context (challenge_name).
• {solution_name} — from the solution context (solution_name).
• {primary_numbers} — comma-separated (N) values from the solution context's paper_nums.
• {papers_input} — contents of <workdir>/papers_input_core.txt (or the per-solution copy).
• {allowed_numbers} — contents of <workdir>/papers_input_core.txt.allowed.txt.

Call the LLM (temperature ~0.2; allow ~12000 max tokens to fit scratchpad + tree). Save the raw response to <workdir>/details/<key>_raw.md.

(c) Parse it (output goes to a sibling parsed/ dir, not details/, so step 13's assemble_report doesn't pick up parse outputs as render outputs):

python scripts/cli.py parse_detail \
    --raw <workdir>/details/<key>_raw.md \
    --context <workdir>/solutions/<key>.json \
    --out <workdir>/parsed/<key>.json

The parsed JSON's edges list is the input to Step 11.

If the host supports concurrent tool calls, fan all per-solution LLM calls + their parse_detail follow-ups out in parallel.

Fan-out task brief (when delegating step 10b to a subagent per solution). The subagent's prompt must be self-contained: do not point the subagent at SKILL.md, do not instruct it to "run paper-graph for solution X," and do not give it any CLI invocation to run. The orchestrator does the substitution itself and hands the subagent only:

• The fully-substituted prompt string (already with {goal}, {challenge_name}, {solution_name}, {primary_numbers}, {papers_input}, {allowed_numbers} filled in).
• The expected response shape: raw Markdown evolution tree per references/detail.md's output spec.
• An explicit instruction: "Call your LLM with the prompt below and return only the raw Markdown response. Do not read any other file, do not run any shell command, do not invoke any other skill."

Do not point the subagent at SKILL.md, do not instruct it to "run paper-graph for solution X", and do not give it any CLI invocation to run. The subagent returns the Markdown text; the orchestrator writes it to <workdir>/details/<key>_raw.md and runs parse_detail itself. A subagent that re-reads SKILL.md will restart the whole workflow from step 1 — the fix is to keep the subagent prompt bounded as above.