Check Reporting

Check-Reporting Skill

Reference Files

Checklists are vendored under ${CLAUDE_SKILL_DIR}/references/checklists/, one file per instrument. Each file's header gives its version, source citation and licence, and says when its item text is an own-words summary rather than the published wording.

• STROBE.md -- STROBE
• STROBE_MR.md -- STROBE-MR 2021
• RECORD.md -- RECORD 2015 (RECORD-PE for drug studies)
• REMARK.md -- REMARK
• TARGET.md -- TARGET 2025
• GATHER.md -- GATHER 2016
• CHEERS_2022.md -- CHEERS 2022
• CROSS.md -- CROSS 2021 + CHERRIES (internet surveys)
• SRQR.md -- SRQR 2014 (all qualitative approaches)
• COREQ.md -- COREQ 2007 (interviews and focus groups)
• STARD.md -- STARD 2015
• STARD_AI.md -- STARD-AI 2025
• TRIPOD.md -- TRIPOD 2015
• TRIPOD_AI.md -- TRIPOD+AI 2024
• TRIPOD_LLM.md -- TRIPOD-LLM 2025
• PGS_RS.md -- PGS-RS / PRS-RS 2021
• CONSORT.md -- CONSORT 2025
• CONSORT_AI.md -- CONSORT-AI 2020
• SPIRIT.md -- SPIRIT 2025
• SPIRIT_AI.md -- SPIRIT-AI 2020
• CLAIM_2024.md -- CLAIM 2024
• DECIDE_AI.md -- DECIDE-AI 2022
• MI_CLEAR_LLM.md -- MI-CLEAR-LLM
• CLEAR.md -- CLEAR (radiomics)
• ARRIVE_2.md -- ARRIVE 2.0
• CARE.md -- CARE 2013
• SQUIRE_2.md -- SQUIRE 2.0
• GRRAS.md -- GRRAS
• PRISMA_2020.md -- PRISMA 2020
• PRISMA_2020_Abstracts.md -- PRISMA 2020 for Abstracts, 12 items. A separate instrument, not a subset of the 27-item checklist (main item 2 defers to it): score it with its own denominator.
• PRISMA_DTA.md -- PRISMA-DTA
• PRISMA_P.md -- PRISMA-P
• PRISMA_ScR.md -- PRISMA-ScR
• MOOSE.md -- MOOSE
• SWiM.md -- SWiM
• AMSTAR2.md -- AMSTAR 2
• QUADAS3.md -- QUADAS-3
• QUADAS2.md -- QUADAS-2
• QUADAS_C.md -- QUADAS-C
• RoB2.md -- RoB 2
• ROBINS_I.md -- ROBINS-I
• ROBINS_E.md -- ROBINS-E
• ROBIS.md -- ROBIS
• ROB_ME.md -- ROB-ME
• RoB_NMA.md -- RoB NMA
• PROBAST.md -- PROBAST
• PROBAST_AI.md -- PROBAST+AI
• NOS.md -- NOS
• COSMIN_RoB.md -- COSMIN RoB

Workflow

Step 0: Existing-checklist staleness pre-check

If a checklist already exists for this project (qc/reporting_checklist.json or a prior .md report), verify it targets the current manuscript before reusing it — one generated against an older version carries stale section/line references and a stale version label:

python3 "${CLAUDE_SKILL_DIR}/scripts/check_checklist_version.py" \
  --checklist qc/reporting_checklist.json --manuscript manuscript_v8.md

A non-zero exit means the checklist is stale (older target_version, changed source_sha256, different target_manuscript) or pre-dates the version contract: regenerate it against the current manuscript (Steps 1–5) rather than reusing it. Every report you generate carries the target_manuscript / target_version / source_sha256 fields (Part A header + Part D JSON) so this check works next round.

Step 1: Select Guideline

Use the guideline the user names; otherwise auto-detect it from the study type. If the study type is ambiguous, ask the user to confirm before selecting a guideline.

Auto-detection mapping:

QUADAS-3 has two protocol-stage phases, and this skill usually runs too late for them. Phase 1 (state the synthesis question) and phase 2 (define the ideal test accuracy trial each judgement is made against) are review-level and belong in the protocol, with the review-specific guidance for answering each signalling question. Reaching them for the first time during manuscript QC means writing the comparator after seeing the results. If they are missing, say so as a limitation rather than reconstructing them, and route the protocol work to /meta-analysis Phase 1. Phases 3–6 are what a QC pass can genuinely run.

Rules:

• A guideline the user names is the one you score. The rules below choose the guideline only when the user names none. If they call for a different or additional instrument (STARD-AI for an AI index test, TRIPOD+AI for an ML prediction model, an AI extension), score the named guideline and put one line at the top of the report saying which instrument fits better and the command to rerun, e.g. /check-reporting STARD-AI. Do not switch or add instruments on your own: the user maps the report to the checklist form the journal asked for.
• If the study involves AI/ML, apply the AI version of the guideline. CONSORT-AI and SPIRIT-AI are scored alongside their base guideline; TRIPOD+AI, STARD-AI and TRIPOD-LLM replace theirs (below).
  • Exception — TRIPOD: TRIPOD+AI 2024 (Collins et al., BMJ 2024) is a complete rewrite, not an addendum to TRIPOD 2015 (Moons et al., Ann Intern Med 2015). For non-AI prediction models, use TRIPOD 2015 only. For AI/ML prediction models, use TRIPOD+AI 2024 only. Do NOT apply both simultaneously.
• STARD-AI (Sounderajah et al., Nat Med 2025) extends STARD 2015 with 14 new and 4 modified items (40 total) and incorporates all STARD 2015 items. For AI diagnostic accuracy studies use STARD-AI only — do NOT apply STARD 2015 and STARD-AI simultaneously.
• TRIPOD-LLM (Gallifant et al., Nat Med 2025) is the reporting guideline for studies that develop, fine-tune, prompt, or evaluate a large language model for a clinical/biomedical task. It extends the TRIPOD family (TRIPOD 2015 → TRIPOD+AI 2024 → TRIPOD-LLM 2025); name the base instrument and the extension and cite each. It is modular — task-specific items (Annotation, Prompting, Summarization, Instruction-tuning) are N/A when that component is absent. Use TRIPOD-LLM for LLM studies in place of TRIPOD+AI; pair with MI-CLEAR-LLM when LLM accuracy is an evaluated outcome. The vendored checklist is an educational summary (own-words paraphrase of item intent); complete the official instrument for a submission checklist.
• MI-CLEAR-LLM is a supplementary checklist (8 item categories in the 2025 update; the 2024 original had 6), not a standalone reporting guideline. Always pair it with the study's primary guideline (e.g., STARD-AI for AI diagnostic accuracy, CLAIM for imaging AI). Apply it whenever the study evaluates LLM accuracy as an outcome — do NOT apply it merely because the manuscript was written with LLM assistance. Its scope is LLM accuracy studies (including VLMs interpreting images); it does not apply at study level when a generative model produces the images under study (next bullet).
• Generative-AI images as the study object (a generative model synthesizes images and the study evaluates their realism, controllability, real-vs-synthetic distinguishability, or model-vs-model quality) has no single dominant checklist. Assemble: CLAIM 2024 (imaging-AI umbrella; model-development items N/A when commercial models are used as-is) + FUTURE-AI traceability + MI-CLEAR-LLM transparency items only (prompt/model/version/params/runs — for generation provenance, not study-level compliance) on the generator side; STARD-AI (for real-vs-synthetic detection) + GRRAS (reader reliability) + MRMC reporting on the evaluation side. Map applicable items and cite base + extension; never claim wholesale compliance. Full decision aid: ${CLAUDE_SKILL_DIR}/references/genai_image_study_object_decision_aid.md.
• If multiple guidelines apply (e.g., a diagnostic accuracy study that is also an AI study), check against all relevant guidelines and merge into one report.

Step 2: Load Checklist

1. Run the fail-fast guard first for every guideline you intend to apply:

   python "${CLAUDE_SKILL_DIR}/scripts/check_checklist_exists.py" --guideline "STARD-AI"

   • Exit 0 → the vendored checklist exists; read it from ${CLAUDE_SKILL_DIR}/references/checklists/ and proceed.
   • Exit 1 (MISSING_CHECKLIST_CONTRACT_VIOLATION) → the guideline is routed but no checklist file is vendored. Do not construct items from memory. Halt, report the violation to the user, and stop unless they explicitly opt in (next bullet).
   • Exit 2 (UNKNOWN_GUIDELINE) → the name is not recognised; confirm the correct guideline with the user.

2. No silent fallback. A from-memory checklist is permitted only when the user explicitly accepts it — re-run the guard with --allow-from-memory (exit 0 + a NON-AUTHORITATIVE warning). The report MUST then carry a prominent banner that the assessment was constructed from model knowledge and is not backed by a vendored checklist, and submission_safe must not be asserted on its basis.

Step 3: Scan Manuscript

Read the whole manuscript before assessing any item — including tables, figures and their captions, supplementary material, and the reference list (registration numbers and protocol references often sit there).

Step 4: Assess Each Item

Items most often missing in medical manuscripts — look for these first, whichever guideline applies: registration number and registration/amendment date consistency (run Step 4c), sample-size justification, missing-data handling, blinding, funding and conflicts of interest, ethics approval with committee name and approval number, and a data availability statement; for AI studies, the training/validation/test split, model architecture and hyperparameters, failure-mode analysis, fairness/bias assessment, and commercial interests with data/code availability.

For every checklist item, determine:

For each item, record:

• Status: PRESENT / PARTIAL / MISSING / N/A
• Location: Section name and paragraph or approximate position (e.g., "Methods, paragraph 3")
• Notes: What was found (if PRESENT/PARTIAL) or what should be added (if MISSING)

Be strict. PARTIAL means the item is mentioned but lacks specificity; a vague reference does not count as PRESENT — the detail level must match what the guideline expects. "We used appropriate statistical tests" = PARTIAL (which tests?); "We used the Mann-Whitney U test for continuous variables and Fisher's exact test for categorical variables" = PRESENT. If an item is genuinely unclear in its applicability, mark it N/A with justification.

Two gaps that are easy to pass:

• Power-aware framing of a null result (STROBE 16a / 18 / 20) — for an observational study whose headline is a non-significant association, a flat "X was not associated with Y" overreads the data when the analysis is not powered to exclude a clinically meaningful effect. Mark item 18/20 PARTIAL unless the manuscript states the precision as an exclusion (e.g., "the 95% CI excluded an eGFR difference larger than ~1.7") or reports a minimum detectable effect — "no effect" vs "could not exclude an effect of size X" are different claims, and a negative conclusion needs the latter.
• Confounder-selection rationale, not "adjust for everything that differs" (STROBE 16a explicitly asks which confounders were adjusted for and why) — flag a kitchen-sink adjustment set chosen because variables differ in Table 1. The Methods must give a causal rationale (DAG / prior literature) and must not adjust for a mediator or consequence of the outcome (over-adjustment, e.g. serum uric acid in an eGFR model); both an unjustified inclusion and an unjustified omission are item-16a gaps.

What is appraised is the source paper's reporting — never your convenience in using it. This holds for every instrument here, reporting checklists and risk-of-bias / quality tools alike, and is easiest to lose in a systematic review, where you read each paper in order to extract from it. An item asking "are the results clearly reported?" is not asking "were they reported in the unit my pool needs". If a downgrade's stated reason turns on a denominator, an analysis unit, a subgroup you needed and they did not report separately, or a format you could not parse, it is an extraction note, not a scoring reason: record it in a separate extraction-note column and restore the score. An extraction limitation often belongs in your limitations paragraph; folded into the score it makes the appraisal unreproducible, because another assessor with a different pool would score the same paper differently.

Step 4b: Section Boundary Check

In addition to checklist items, verify that:

• Results section contains only factual findings: no interpretation, no "why" explanations, no prior literature comparisons, no evaluative adjectives without numbers.
• Discussion section does not introduce new data not presented in Results.
• Flag any boundary violation as a separate finding in Part C Action Items with the label [BOUNDARY].