Paper OCR Notes Pipeline

Paper OCR Notes Pipeline

This skill is a reusable policy for paper ingestion and note production. It must stay self-contained and work in any workspace.

Do not require any companion README.md or local notes file to run this workflow. If one exists, treat it only as supplemental context.

Objective

Generate a self-contained notes package from a paper source while preserving provenance, factual grounding, OCR quality, and teachability.

Inputs

• One of: a PDF path, a paper URL, or an arXiv abs URL.
• Optional: preferred folder slug.
• Optional: output root directory.

Output Root Resolution

Use this priority order:

1. If the user provides an output root, use it.
2. Else create and use ./papers-articles in the current working directory.

Output Contract

Create a dedicated folder:

<output-root>/<paper-slug>/

Required files:

• source file: <paper>.pdf or link.txt for URL-only sources,
• notes.md as the primary deliverable,
• README.md as a paper-local artifact and provenance index,
• <paper>.ocr.feedback-loop.txt as the final OCR output,
• <paper>.ocr.feedback-loop-comparison.md as the OCR replacement report.

notes.md must be sufficient for a new reader to learn the paper without opening extra files.

Canonical Notes Sections

Use these headings in order:

1. # <Paper Title>
2. ## TL;DR
3. ## Beginner's Guide
4. ## Paper Summary
5. ## Deep Dive
6. ## Critical Analysis
7. ## Learning Path
8. ## Key Equations Summary
9. ## References

Nested Skills

• Use pdf-ocr-feedback for pagewise OCR refinement.
• Use technical-writing-workflow while drafting and revising notes.md.

Core Principles

1. Reader intuition comes before technical detail.
2. Do not blur distinct concepts.
3. Evidence-first writing: claims point to table, figure, section, or page evidence.
4. Revision is mandatory.
5. Identity verification is mandatory before trusting extracted content.
6. Uncertainty and limitations must be stated explicitly.

Workflow

Follow the phases in order.

Phase 0 - Intake and setup

1. Resolve canonical identity: title, authors, date/version, identifier (arXiv/DOI when available).
2. Create output folder and store source.
3. Initialize notes.md skeleton with required section order.
4. Create paper-local README.md listing source and produced artifacts.

Phase 1 - Extraction and OCR feedback loop

Use nested skill pdf-ocr-feedback for pagewise OCR refinement.

Required OCR behavior:

1. Pass-1 extraction across all pages.
2. Per-page quality scoring and weak-page detection.
3. Pass-2 retries only on weak pages.
4. Merge pages in original order.

Required OCR format:

• Preserve ===== PAGE N ===== delimiters.
• Preserve page order.
• Record replaced pages and rationale in comparison report.

Phase 1.5 - Identity gate (mandatory)

Before trusting extracted content, verify anchors:

• title matches target,
• author block matches reasonably,
• identifier/date lines are consistent when available.

If any anchor mismatches:

1. reject extraction,
2. rerun extraction,
3. revalidate before proceeding.

Phase 2 - Ground-truth extraction (facts only)

Collect source-backed facts only (no interpretation yet):

• problem statement and failure mode,
• contributions as stated by authors,
• method overview,
• key equations and definitions,
• experimental setup: tasks, datasets, models, baselines, metrics, ablations/stress tests,
• headline quantitative results,
• limitations stated by authors.

Stop condition: answer "what did they do" without guessing.

Phase 3 - Compose notes.md in strict order

1. TL;DR (problem, key idea, 1-2 evidence numbers, caveat)
2. Beginner's Guide (college-level, define symbols once, one concrete toy example)
3. Paper Summary (problem recap, contributions, results, experiments table)
4. Deep Dive (definitions, derivation steps, implementation sketch, optional pseudocode)
5. Critical Analysis (assumptions, objections, failure modes, validation/refutation experiments)
6. Learning Path (prereqs, next study steps, diagnostics)
7. Key Equations Summary (equation + one-line meaning)

While drafting, apply technical-writing-workflow:

• anchor explanations in familiar ground,
• define terms at point of need,
• keep ontology and score ownership clear,
• cut implementation-operator detail unless it matters scientifically,
• keep claims tied to evidence.