Find Paper References

Source: https://github.com/aipoch/medical-research-skills

find-paper-references — Academic Paper PubMed Reference Finder

Tools

Scripts are located in the scripts/ subdirectory of this skill:

• batch_search.py — Main entry point: accepts all queries at once, parallel search, batch esummary fetch, typical runtime 10-20s
• remap_refs.py — Converts [PMID:XXXXXXXX] markers to [1][2] numbering and generates formal references (not called by this skill — handled by subsequent endnote/zotero skill)
• convert_to_docx.py — Converts .md to .docx (not called by this skill)

Skill Boundary

┌─────────────────────────────────────────────────────┐
│  find-paper-references                              │
│  ─────────────────────────────────────────────────  │
│  Step 0-3: Identify knowledge points → Build PubMed search JSON │
│  Step 4:   batch_search.py batch search                         │
│  Step 5:   Select articles → Insert [PMID:xxxxxxxx] markers     │
│  Step 6:   Write to .md file                                    │
│  Step 7:   Generate _candidates.md candidate reference list     │
│  Step 8:   Prompt user to choose EndNote / Zotero to continue   │
│  ──────────────── Workflow ends ──────────────────────────────── │
│                                                     │
│  Formatting → format-references-endnote (EndNote)              │
│            → format-references-zotero (Zotero)                 │
└─────────────────────────────────────────────────────┘

Workflow (Execute in Order)

Step 0: Ask for NCBI API Key

Before starting any search, check and ask:

Do you have an NCBI API key? If so, search speed increases from 3 req/s to 10 req/s.
No worries if not — just skip and we'll proceed.

• User provides key: Before all script calls in this session, execute:

  # Windows
  $env:NCBI_API_KEY = "user_provided_key"
  # Mac/Linux
  export NCBI_API_KEY="user_provided_key"

  Then continue the workflow normally. Key is valid only for this session, not written to any file.

• User doesn't have one / skips: Proceed to Step 1. The script automatically uses the conservative 3 req/s rate.

Step 1: Determine Target File + Article Type

Read the full file content, then determine article type — can be auto-detected or confirmed with the user:

Auto-detection rules:

• Contains ## Materials and Methods / ## Methods section → Research Article
• No such section, body consists mainly of review-style paragraphs → Review
• When uncertain, ask the user directly

Article Type Citation Rules

Based on the detected type, this workflow must follow the corresponding rules:

Research Article Rules

Citation zones:  Introduction and Discussion only
            Materials/Methods and Results → skip entirely, insert no citations
Per knowledge point: max 2 articles (top 2 by relevance, no padding)
Total target: at least 30 unique PMIDs in the text

Review Rules

Citation zones:  All paragraphs except Conclusion
            Conclusion paragraphs → skip
Per knowledge point: max 5 articles
Total target: approximately 1 citation per 100 words, +/- 20%
            Target range = [word_count*0.8/100, word_count*1.2/100] (rounded)
            Example: 5000 words → target 40-60 articles; 8000 words → target 64-96 articles

Step 2: Identify Sentences Requiring Citations

Read through the paper, strictly following the citation zone rules for the current article type, skip sections that don't need citations.

Within allowed citation zones, identify all knowledge points requiring literature support:

Needs citation (applicable to allowed sections):

• Epidemiological data ("Lung cancer is one of the most common malignancies worldwide")
• Known biological functions of proteins/genes ("GPX4 reduces lipid peroxides to non-toxic alcohols")
• Signaling pathway descriptions ("xCT mediates cellular cystine uptake")
• Published research conclusions ("TRIM3 is downregulated in breast cancer")
• Known mechanisms of treatments/drugs ("Erastin induces ferroptosis by inhibiting xCT")

Never needs citation:

• This study's own experimental results
• Instrument/reagent descriptions in Methods (entire section skipped for Research Articles)
• Results section (entirely skipped for Research Articles)
• Conclusion section (skipped for Reviews)
• General technical operation descriptions

After identification, estimate total knowledge points:

• Research Article: target ≥ 30 unique citations → typically need to identify 20-35 knowledge points
• Review: calculate total word count first, estimate target citations at 1 per 100 words, typically need knowledge points close to target number (1-2 articles per point)

Organize identified knowledge points into a list, each containing:

• Section name (to confirm it's in an allowed citation zone)
• The sentence from the original text
• Core concept of the knowledge point (for generating search terms)

Step 3: Organize All Queries into Temporary JSON

Organize all knowledge points into the following format, write to %TEMP%\ref_queries.json (or /tmp/ref_queries.json):

[
  {"id": 1, "description": "global lung cancer incidence", "query": "global cancer statistics 2020 GLOBOCAN lung cancer Sung", "method": "pubmed"},
  {"id": 2, "description": "ferroptosis definition", "query": "Ferroptosis iron-dependent nonapoptotic cell death Dixon 2012", "method": "pubmed"},
  {"id": 3, "description": "TRIM3 structure and function", "query": "TRIM3 ubiquitin ligase RING domain substrate degradation", "method": "litsense"},
  ...
]

Method options:

• "pubmed" — Keyword-based precise search (recommended for most knowledge points)
• "litsense" — Semantic search (suitable for descriptive sentences, specific protein functions, etc.)
• "auto" — Try LitSense first; if results < 2 articles, automatically fall back to PubMed

Query writing guidelines:

• Use English, 3-8 keywords
• Prefer MeSH terms (ferroptosis, non-small cell lung cancer, ubiquitin ligase)
• For known classic papers, add author or year to query ("Dixon 2012", "Stockwell 2017")

Step 4: Run batch_search.py Once to Get All Results

python "SKILL_DIR/scripts/batch_search.py" "%TEMP%\ref_queries.json" --max 5

Script searches all queries in parallel, batch-fetches esummary, typical runtime 10-20 seconds (vs. minutes for sequential searching).

Output JSON format:

[
  {"id": 1, "description": "...", "results": [{pmid, title, authors, journal, year, doi}, ...]},
  ...
]

If any result is empty, create a new JSON file with adjusted keywords and re-run batch_search.py.

Step 5: Select Most Relevant Articles and Insert PMID Markers

For each sentence requiring citation:

1. Sort candidates by relevance, select according to article type limits:
   • Research Article: max 2 per knowledge point, select only the most relevant
   • Review: max 5 per knowledge point, can cover different research angles
2. Insert [PMID:XXXXXXXX] at end of original sentence; for multiple: [PMID:111][PMID:222]
3. If same PMID already cited, reuse it — do not insert duplicates

After insertion, count current unique PMIDs:

• Research Article: if fewer than 30, return to Step 2 to identify more knowledge points
• Review: calculate word count, compute target range [words0.8/100, words1.2/100] (rounded), if not within range, return to Step 2 to add or reduce

Example (before/after):

Before: Ferroptosis is a form of regulated cell death driven by iron-dependent lipid peroxidation.
After:  Ferroptosis is a form of regulated cell death driven by iron-dependent lipid peroxidation[PMID:25789077].

Flexible formats supported by remap_refs.py (all handled correctly):

• Case insensitive: [PMID:123] [pmid:123] [Pmid:123]
• Spaces around colon: [PMID: 123] [ PMID : 123 ]
• Multiple in one bracket: [PMID:123, PMID:456] → [1][2] (spaces and comma variants accepted)