Manuscript Provenance

Manuscript Provenance Audit

Pipeline position: Phase 2a (grounding audit). Runs in parallel with manuscript-typography. Depends on: content settled after Phase 1 fixes. Produces macro manifest consumed by manuscript-review Pass 13 (Cross-Element Coherence).

Purpose

Verify that a manuscript is a faithful rendering of computational outputs. Every number, table, figure, category label, ordering, and threshold in the document must trace to a specific script, config file, or pipeline output. Manual data entry in a manuscript is a reproducibility defect.

This skill produces a provenance map — a structured report linking each manuscript artifact to its generating code — and flags every break in the chain.

Companion skill: manuscript-review audits the document as prose (structure, argumentation, citations). This skill audits whether the document content is computationally grounded. Run both for complete pre-publication coverage.

Boundary Agreement with manuscript-review

Rule: This skill never judges prose quality. manuscript-review never opens the codebase. Each reads the other's report when available.

Integration point — Macro Manifest: This skill produces a macro manifest as part of the §4 audit: a structured list of every macro-injected value with:

• Macro name (e.g., \bestf)
• Resolved value (e.g., 0.847)
• Source (script + output file that generates it)
• Location(s) in manuscript text (file, line number, surrounding sentence)
• Classification (TRACED / MACRO-TRACED / CONFIG-TRACED / UNTRACED / STALE)

manuscript-review's Pass 13 (Cross-Element Coherence, §24) consumes this manifest to check whether the prose surrounding each injected value is appropriate for the actual numeric value. Provenance owns "is this value computationally grounded?" Review owns "does the text wrapping this value make sense given what the value is?"

Scope

In scope:

• Numbers, metrics, percentages in manuscript text
• Tables (content, ordering, formatting)
• Figures (generation scripts, data sources)
• LaTeX macros (\newcommand, \def, \pgfmathsetmacro)
• Terminology, mode names, mechanism labels, category names
• Ordering of items in enumerations, tables, discussion
• Config values (thresholds, hyperparameters, model names)
• Pipeline completeness (raw data → final PDF)
• Timestamp consistency (scripts vs outputs)

Out of scope:

• Prose quality (→ manuscript-review)
• Citation hygiene (→ manuscript-review)
• Argumentation structure (→ manuscript-review)
• Code quality/style (separate concern)

Inputs

This audit requires TWO artifacts:

1. Manuscript source — LaTeX .tex files (preferred), or PDF/DOCX as fallback
2. Codebase — the scripts, configs, and pipeline that generate manuscript content

If the user provides only one, ask for the other. LaTeX source is strongly preferred over compiled PDF — provenance auditing requires seeing the raw markup, macros, and input commands.

Workflow

Phase 1 — Inventory

1a. Manuscript Artifact Extraction

Read all .tex files (main + included via \input/\include). Extract:

• Inline values: bare numbers in running text (percentages, counts, metrics, p-values, confidence intervals, thresholds, sizes)
• LaTeX macros: all \newcommand, \def, \pgfmathsetmacro, and custom command definitions that carry data values
• Tables: full content of every tabular/table environment — cell values, row/column ordering, headers
• Figures: \includegraphics paths, caption content, referenced data
• Input files: any \input{generated/*.tex} patterns that pull from script-generated LaTeX fragments
• Labels and references: \label/\ref pairs for cross-referencing
• Terminology: named modes, mechanisms, strategies, categories, method names used in prose
• Ordered lists: any enumerated or ranked items (methods compared, features listed, results ordered)

Build an artifact registry — a flat list of every data-carrying element in the manuscript with its location (file, line number).

1b. Codebase Mapping

Scan the project directory. Identify:

• Pipeline entry points: Makefile, snakemake, dvc.yaml, run.sh, main.py, or equivalent orchestration
• Analysis scripts: files that produce numbers, tables, figures
• Config files: config.toml, config.yaml, .env, params.yaml, hyperparameter files
• Output directories: where scripts write results (results/, output/, figures/, tables/, generated/)
• Generated LaTeX fragments: .tex files in output directories that scripts produce for \input inclusion
• Data files: CSVs, JSON, HDF5, pickles that intermediate results flow through

Build a source registry — a flat list of every code artifact that produces or configures manuscript content.

Phase 2 — Provenance Tracing

For each entry in the artifact registry, attempt to establish a provenance chain: manuscript value → generated output → script → input data/config.

2a. Value Provenance

For every number in the manuscript:

1. Search for the value in script outputs (logs, result files, generated LaTeX)
2. Trace the output back to the script that produces it
3. Verify the script reads from data/config (not hardcoded)
4. Record the full chain or flag as UNTRACED

Classification:

• TRACED — full chain from manuscript value to generating code
• MACRO-TRACED — value defined in a LaTeX macro that is generated by a script
• CONFIG-TRACED — value comes from a config file read by scripts
• UNTRACED — no provenance chain found; manually entered
• STALE — provenance chain exists but output is older than generating script

2b. Table Provenance

For each table:

1. Is the table content generated by a script (CSV → LaTeX, or direct LaTeX generation)?
2. Is the row/column ordering determined by code (sorted by metric, alphabetical, grouped by category) or manually arranged?
3. Are header labels matching code-defined names?
4. Are formatting choices (bold for best, significant figures) applied by code?

Classification:

• GENERATED — entire table produced by script
• PARTIAL — some cells generated, some manual
• MANUAL — no generation script found
• ORDER-MANUAL — content generated but ordering is manually set

2c. Figure Provenance

For each figure:

1. Does a script produce the exact file referenced by \includegraphics?
2. Does the script use a deterministic seed for reproducibility?
3. Is the figure output path in the script consistent with the LaTeX reference?
4. Are figure parameters (colors, labels, axis ranges) set in code or manually edited post-generation?

Classification:

• GENERATED — script produces the exact file
• POST-EDITED — script generates base figure, but manual edits detected (e.g., Illustrator metadata, different checksum than script output)
• MANUAL — no generating script found
• STALE — generating script modified after figure file

2d. Terminology Provenance

For each named mode, mechanism, category, or method label:

1. Is the term defined in code (enum, constant, config key, class name)?
2. Does the manuscript term match the code term exactly?
3. If the manuscript uses a display-friendly name, is there an explicit mapping in code or config?

Classification:

• CODE-DEFINED — term matches code definition
• MAPPED — explicit code→display mapping exists
• UNMAPPED — term appears in manuscript but not in code
• INCONSISTENT — term appears in both but differs (e.g., code says greedy_search, manuscript says "Greedy Search" in some places and "greedy approach" in others)

2e. Ordering Provenance

For each ordered list, ranked comparison, or sequenced enumeration:

1. Does code determine the ordering (sort by metric, alphabetical, enum order)?
2. Does the manuscript ordering match the code-determined order?
3. Are there items in the manuscript list not present in code output, or vice versa?

Classification:

• CODE-ORDERED — ordering matches code output
• MANUAL-ORDER — ordering differs from code output or no ordering logic in code
• SUBSET-MISMATCH — manuscript lists different items than code produces