Self Review

Self-Review Skill

Check the user's own manuscript before submission and produce an actionable list of anticipated reviewer comments, each with a specific fix — not a written review.

Optional Flags

• --fix: after the report, apply fixes for every issue with fixable_by_ai: true, editing the manuscript in place, then report a diff summary. Never fix fixable_by_ai: false issues (missing data, design flaws). Maximum 2 fix-and-re-review iterations; if the score is still below threshold after the second, stop and report what remains as structural (inside /write-paper this routes to Phase 7.4a Audit Recovery).
• --json: also emit the structured JSON block (Phase 3c). Default when called from /write-paper Phase 7.
• --panel: run the multi-agent panel review (Phase 2.6) — domain-expert reviewers in parallel plus an editor synthesis — instead of the single-pass review. Opt-in and off by default, because it spawns N reviewer agents + 1 editor and costs several times more tokens; reserve it for a high-stakes final pass on a top-tier target. Do not combine with --fix: a panel diagnoses and prioritizes; run --fix as a separate pass once the author has triaged the panel's findings.

Severity Framing

• Fatal: a design flaw existing data cannot fix (data leakage that invalidates all results, no reference standard, label–feature circularity). Submission would likely be rejected. Reserve Fatal for true design-level problems.
• Fixable: significant but addressable with existing data (missing calibration, unclear exclusion criteria, absent CIs, incomplete reporting). Most issues are Fixable.

Two Objectives: the Floor and the Ceiling

The floor (categories A–K and the gates of Phases 2.5–2.5f) minimizes rejection-for-cause — fabricated citations, numbers that do not reconcile, overclaims, missing checklist items, leakage — and much of it works by adding a hedge, caveat, disclosure or audit trail. Iterated, that over-hardens a manuscript: every finding is correct, yet the whole reads as a defensive audit (over-hedged, audit trail in the body, Abstract buried under caveats, the strongest robustness result hidden in Limitations). The ceiling pass (category L / Phase 2.5g) runs after the floor gates, reads the accurate manuscript as a whole, and recommends only SUBTRACTION — REMOVE, MOVE, or TIGHTEN. It is advisory, never blocks, and cannot relax a floor gate. Report its findings as their own block (Phase 3), not folded into the "add this" comments. Phase 2.5i then reads the floor + ceiling state to declare the loop done, including a zero-edit PASS.

Workflow

Phase 1: Intake

1. Get the manuscript — PDF, Word doc, or pasted text.

2. Ask the user: target journal (sets reporting standards and scope); manuscript type (original research / review / perspective / technical note / letter / meta-analysis / case report); anything they are already worried about. On an interactive run, offer the --panel review (Phase 2.6) once, in one line, then proceed single-pass unless the user opts in. Never offer or apply the panel under --json or when called from /write-paper.

3. Read the full manuscript.

4. SSOT gate — confirm there is one manuscript, not several. Self-review reads a single file, so drift between a legacy working copy and the live submission copy is invisible to it. Before a --panel run or any pre-submission pass, check for multiple copies:

   find . \( -path '*manuscript*' -o -path '*main_document*' \) -name '*.md' | grep -v node_modules

   If more than one manuscript-like file exists, confirm which is the SSOT and run /sync-submission's divergence gate — a STALE_COPY (an SSOT numeric claim or heading that did not propagate to the other copy) is a P0 that must clear first:

   python3 "${CLAUDE_SKILL_DIR}/../sync-submission/scripts/detect_copy_divergence.py" \
     --ssot <ssot>.md --copy <other-copy>.md

   Review the SSOT copy, never a stale one. Under --panel this is blocking: if find returns more than one file and the SSOT is not pinned (no SSOT.yaml with truth.manuscript_md, no explicit --ssot <path>), STOP before spawning any reviewer and have the user name the SSOT (and clear any STALE_COPY), because a panel on a stale copy wastes the whole pass. Do not auto-pick the longest/newest file. A single-pass review may proceed on the one file it was given.

Phase 2: Systematic Check

Work every category the Research-Type Adaptation table (below) marks as applicable, and for each item decide whether a reviewer would raise it as a Major or Minor comment. The per-item check tables are in references/phases/phase2_systematic_check.md — read it once you have the manuscript and know its type.

Run the deterministic gates at Phase 2 entry, on every path:

# D. endpoint↔conclusion scope
python3 "${CLAUDE_SKILL_DIR}/scripts/check_scope_coherence.py" \
  --manuscript manuscript.md --out qc/scope_coherence.json --strict

# J. classical-style body conventions
python3 "${CLAUDE_SKILL_DIR}/scripts/check_classical_style.py" \
  --manuscript manuscript.md --out qc/classical_style.json --strict

# K. reviewer-team consistency (SR/MA only; pass the extraction JSON file or directory)
python "${CLAUDE_SKILL_DIR}/scripts/check_reviewer_team_consistency.py" \
    --manuscript manuscript.md --prospero prospero/record.md \
    --extraction-json extraction/ --out _audit_self/reviewer_team_consistency.md

# J/D. Perspective structure (genre-gated: silent unless article_type is a Perspective).
# Pass the known type via --type; it also self-detects from the front-matter article_type.
python3 "${CLAUDE_SKILL_DIR}/scripts/check_perspective_structure.py" \
  --manuscript manuscript.md --type "${TYPE:-}" --out qc/perspective_structure.json

# Was every reported analysis ever defined (outcome, reference standard)?
python3 "${CLAUDE_SKILL_DIR}/scripts/check_analysis_definitions.py" \
  --manuscript manuscript.md --json --strict > qc/analysis_definitions.json

Verdict mapping: CROSS_SECTIONAL_PROGNOSTIC, SURROGATE_CARE_DIRECTIVE, SECTION_SYMBOL, INBODY_AI_DISCLOSURE, any reviewer-team hit (exit 1), MODEL_OUTCOME_UNDEFINED (a Cox / Fine–Gray / logistic model with no outcome named), MODEL_NOT_IN_METHODS, and REFERENCE_STANDARD_UNDEFINED (discrimination or calibration with nothing to score against) are Anticipated Major Comments. CROSS_SECTIONAL_YIELD_LANGUAGE, ELIGIBILITY_PROSE, DECIMAL_INCONSISTENCY, EM_DASH_OVERUSE, PERSPECTIVE_HEADING_NOT_ASSERTION, PERSPECTIVE_ABSTRACT_NO_AUTHORIAL_MOVE, and TIER_LABEL_UNDEFINED are Minor. The per-verdict rationale and resolution paths are in the Phase 2 reference file.

ANALYSIS_LOAD is informational, never a verdict: many analyses are the cause of omitted definitions, not the defect. Do not cut analyses to satisfy it; restore the definitions they crowded out, and if load is genuinely high, move the defensive analyses to the supplement.

Research-Type Adaptation

*Meta-analysis: replace C with heterogeneity (I², prediction intervals), publication bias (funnel plot, Egger), and sensitivity/subgroup analyses.

Type-specific additional checks:

• Observational: confounding (DAG or adjustment strategy), selection bias, exposure measurement validity. Run Phase 2.5e (Confounding Completeness), then apply the O-probes in references/domain-probes/observational_confounding.md — O1 (covariate imbalanced by exposure in Table 1 yet absent from the adjustment set) and O8 (records > subjects with the analysis unit undisclosed; check_cohort_arithmetic.py --id-col) are the deterministic ones, and O7 (adjusting for a mediator/consequence of the outcome) is their opposite-direction twin. For a clinical prediction model (TRIPOD / TRIPOD+AI, nested predictor-set comparison), also apply the CP-probes in references/domain-probes/clinical_prediction_model.md.
• Educational: learning-outcome measurement validity, Kirkpatrick level, control-group adequacy, curriculum fidelity.
• Meta-analyses: search comprehensiveness (2+ databases), screening reproducibility (2 reviewers), per-study RoB, GRADE certainty.
• Case reports: diagnostic-reasoning transparency, timeline completeness, informed consent, generalizability disclaimer.
• Surgical: learning curve, surgeon volume/experience, complication grading (Clavien-Dindo), operative detail.

Domain probe modules — the same probes /peer-review uses, vendored here. Load every module whose row matches:

Apply each probe as an additional source of comments, complementing (not replacing) categories A–K: a conclusion-threatening or design-level finding becomes a Fatal Anticipated Major Comment, a reporting-level finding a Fixable Anticipated Minor Comment, each tagged with the closest category letter (A–K).

For a classifier / NLP / tabular ML manuscript, also run the feature-selection-leakage gate — a data-driven selection (feature selection, univariate filtering, vocabulary, a threshold) fit on the FULL dataset before cross-validation inflates the CV metric:

python3 "${CLAUDE_SKILL_DIR}/scripts/check_cv_leakage.py" \
  --manuscript manuscript.md --out qc/cv_leakage.json

CV_SELECTION_LEAKAGE (Major) fires when a selection token co-occurs with cross-validation and no fold-nesting is disclosed ("within each fold" / "nested CV" suppresses it). Patient-vs-image split leakage is a different check (model-assessment/check_split_leakage.py).

Phase 2.5: Numerical Cross-Verification (Internal)

Before the report, verify internal consistency:

1. Abstract vs Body: every Abstract number matches Results and Tables.
2. Table vs Text: sample sizes, primary outcomes and p-values agree between tables and narrative.
3. Figure vs Text: figure legends match the data described in Results.
4. Percentage arithmetic: n/N percentages are correct (23/150 = 15.3%, not 15.0%). The gate holds each cell to its printed precision (half a unit in the last printed place).
5. CI plausibility: confidence intervals are reasonable for the sample sizes.
6. Rate back-calculation: every rate inverts to its own numerator/denominator (incidence rate ≈ events / person-years × scale, ±rounding). A rate that does not recompute, or implies more events than the cohort can supply, is a Major.
7. Exclusion-cascade and complete-case arithmetic (cohort/observational): start N − Σ(exclusions) == final analytic N, and total − missing == complete. A footnote N that does not equal the subtraction is a Major.

For cohort/observational manuscripts, run the gate instead of eyeballing it (it parses prose equations + GFM tables, and recomputes from a committed CSV when given one):

python3 "${CLAUDE_SKILL_DIR}/scripts/check_cohort_arithmetic.py" \
  --manuscript manuscript.md --data analysis/cohort.csv --id-col mockid \
  --out qc/cohort_arithmetic.json --strict

RATE_BACKCALC / CASCADE_SUM / PARTITION_OVERLAP are Anticipated Major Comments (category A). On health-screening / EMR / registry data pass --id-col (or let it auto-detect a subject-ID column) so it also checks the analysis unit: records > unique subjects with neither the unit nor a one-record-per-subject sensitivity stated emits ANALYSIS_UNIT_UNDISCLOSED (Major — non-independent observations give anti-conservative CIs; probe O8). Flag any remaining internal-consistency discrepancies as Anticipated Minor Comments (category F).

Then recompute what a reviewer recomputes by hand:

# Every "n (%)" in a table, recomputed against its own denominator.
python3 "${CLAUDE_SKILL_DIR}/scripts/check_table_percentages.py" \
  --manuscript manuscript.md --out qc/table_percentages.json --strict

# Every reported P beside a 2×2 (or r×c) count, recomputed from the counts themselves.
python3 "${CLAUDE_SKILL_DIR}/scripts/check_reported_p_from_counts.py" \
  --manuscript manuscript.md --json --strict > qc/reported_p.json

# Every t(df) / F(df1, df2) / χ2(df) / z with a P in its sentence, P recomputed from the statistic.
python3 "${CLAUDE_SKILL_DIR}/scripts/check_test_statistic_p.py" \
  --manuscript manuscript.md --json --strict > qc/test_statistic_p.json   # add --grim grim.json

# Diagnostic-accuracy only: sensitivity/specificity against the reference-standard denominators.
python3 "${CLAUDE_SKILL_DIR}/scripts/check_dta_denominators.py" \
  --manuscript manuscript.md --json --strict > qc/dta_denominators.json

PERCENT_MISMATCH, P_NOT_REPRODUCIBLE, and DTA_DENOMINATOR_MISMATCH / STAGE_ROWSUM are P0 Major — not a rounding disagreement: one of the two numbers is wrong. Run the first two on every manuscript with a table; the third only on diagnostic-accuracy work.

check_test_statistic_p.py reads the statistic and the P at their printed precision: a P no statistic in the rounding interval can give is P_STAT_INCONSISTENT (Minor), and Major P_STAT_DECISION_ERROR when the two also fall on opposite sides of alpha (--alpha). Adjusted or non-standard P values (corrected, exact, Welch, …) and results that match only the unstated one-sided P are never Major. --grim takes declared means ([{"label", "mean", "n", "items"?, "decimals"?}]): GRIM_INCONSISTENT (Major) when no integer sum gives the mean; GRIM_NOT_ASSESSED when n·items >= 10^decimals.

To check each P against its own test, declare the tests as p_tests.json (copy ${CLAUDE_SKILL_DIR}/templates/p_tests.json; schema in references/p_tests_schema.md) and add --tests p_tests.json. The row's P is recomputed with the declared test (fisher, chi2, chi2_yates). A reported P whose whole printed-precision interval lies on the other side of alpha from the recomputed P is P_ALPHA_CROSSING (Major). An adjusted, paired or other: test, a row whose printed percentage is not count/n (another denominator), or a table of 3+ groups (after dropping a Total column) is P_NOT_ASSESSED (Minor).

Known limits: without --tests, check_reported_p_from_counts.py flags a row only when its reported P differs from the crude 2x2 P by more than one order of magnitude under every test family, and only in tables with at least two count rows. It does not flag a reported < bound (<0.001) that the crude P exceeds, or a P on the other side of alpha, by less than that, nor a lone count row (open finding SR-01 in prose-only mode): the table cannot show whether the P is adjusted, paired or on another denominator. Check such P values against the stated test by hand, or declare them. check_table_percentages.py reads one denominator per column. A cell that misses only at its printed precision (within 0.5 pp) on a footnoted row (Current smoker^a | 23 (15.4%) under n = 150 with 149 known) is a Minor PERCENT_PRECISION_NOTE; an unfootnoted row, or a larger miss, is a PERCENT_MISMATCH. Confirm footnoted rows against the footnote.

Phase 2.5a: Numerical Source-Fidelity Audit (External)

Numbers can be self-consistent everywhere and still wrong at the source; only a traversal back to the primary source catches that. First run the displayed-arithmetic gate — a stated difference must equal the subtraction of its two displayed components at the same precision:

python3 "${CLAUDE_SKILL_DIR}/scripts/check_rounded_delta.py" \
  --manuscript manuscript.md --out qc/rounded_delta.json

ROUNDED_DELTA_MISMATCH (Minor) fires when AUCs shown as 0.79 and 0.82 are reported with a difference of 0.02. A higher-precision pair (0.794 vs 0.816) with a 2-dp delta is legitimate and not flagged.

When to run the external audit: MA revisions, submissions, or when the user says "check against the source", "verify extraction", or "random sample". Skip otherwise.

The audit: draw a stratified sample of 5 numerical claims — always including one comparative-arm value and one revision-introduced number — and trace each through three layers (manuscript → extraction CSV → primary-source page; plus analysis script → CSV where a script produced it). Any mismatch is a Major Comment; one that reverses a direction or crosses a significance boundary is a P0 blocker. Every [VERIFY-CSV] tag is a mandatory audit item regardless of sample size.

Read references/phases/phase2_5a_source_fidelity.md when running the external audit — it has the traversal procedure, recording table, sampling strata, and four prose-judgement rules (hand-entered analysis-script inputs; prose↔table statistic-type mismatch, e.g. a median in the text against a mean in Table 1; stale derived CSVs after a model/adjustment-set change; direction reversals internal consistency cannot see).

Phase 2.5a-2: Design & Power Statistic Provenance

Applies only when the manuscript states a sample-size calculation, a power figure, or a detectable-effect claim. These are computed, not copied, so Phase 2.5a cannot check them: re-derive each from the manuscript's own inputs. A value not reproduced by committed code, or reproducible only by a method the committed script does not implement, is a Major Comment (P0 if a headline claim). Read references/phases/phase2_5a2_design_power.md when the manuscript reports a sample-size / power / MDE calculation.

Phase 2.5b: Screening-Count Reconciliation from ID Sets (SR/MA + observational tier/stratum)

When to run: any SR/MA manuscript revision, at any stage (before Phase 3); or any observational manuscript presenting an ordinal tier / mutually-exclusive stratum split. Skip otherwise. A wrong prose total survives every other pass because Abstract, Methods, Results, captions and supplement all cite it back to each other; only a recount from the ID sets catches it.

A. SR/MA — recount from the ID sets. Derive every study count from the screening TSV and the consensus sheet, never from prose, and list the narrative-only IDs explicitly (turning "10 narrative-only studies" into "2 (IDs 120, 474)"). A derived total that disagrees with the Abstract, Methods, Results, Figure 1 caption or Limitations is a P0 Major, blocking submission; an N → M transition claim not backed by an enumerable ID addition/subtraction set is a Major.

B. Observational tier/stratum. A disjoint partition must satisfy Σ(stratum N) == unique total and Σ(stratum events) == total events. Denominators summing above the unique cohort double-count subjects; every stratum n equal to the grand total is a mis-entry. Confirm the reference (baseline) row of any stratified hazard/odds table is present and labelled.

python3 "${CLAUDE_SKILL_DIR}/scripts/check_cohort_arithmetic.py" \
  --manuscript manuscript.md --data analysis/strata.csv --strict

C. Cross-script cut-point consistency. When one cohort is re-stratified in more than one script, the derived categorical must use one cut definition (same breaks, same right= closure, same labels) — otherwise per-stratum Ns drift while the grand total still reconciles. The same gate covers a derived 0/1 composite rebuilt in a second script with a clause dropped.

python3 "${CLAUDE_SKILL_DIR}/scripts/check_binning_consistency.py" \
  --root analysis --root scripts --strict

PARTITION_OVERLAP, BINNING_DRIFT, and DERIVED_DEF_DRIFT are all P0 Major. Read references/phases/phase2_5b_screening_counts.md when doing the SR/MA ID-set recount or a stratified-cohort recount (set definitions, derivation formulas, reconciliation-block template).

Phase 2.5c: Reference Scans (hallucination + adequacy)

2.5c catches a citation that does not exist or has an invented first author; 2.5c-2 catches a claim with no citation. Both need a bibliography — skip them if there is no refs.bib and no reference list. Run /verify-refs --strict first (these scans read its audit), then the adequacy checker:

python3 "${CLAUDE_SKILL_DIR}/scripts/check_reference_adequacy.py" \
  --manuscript manuscript/manuscript.md --bib "$BIB" \
  --article-type "$TYPE" ${CAP:+--journal-cap "$CAP"} \
  --out qc/reference_adequacy.json --strict

A FABRICATED record or any duplicate_findings[] entry in qc/reference_audit.json is a P0 Major Comment that blocks submission. Read references/phases/phase2_5c_reference_scans.md when the manuscript has a bibliography and you are auditing citations.