Systematic Review

Source: https://github.com/aipoch/medical-research-skills

Systematic Review (Meta Skill)

This meta-skill coordinates a complete systematic review pipeline following PRISMA 2020 guidelines. It integrates multi-database literature searching, structured screening, information extraction, quantitative synthesis, and standardized reporting into a rigorous evidence review workflow by combining three specialized skills.

Workflow

Step 1: Protocol Development

Define the review protocol before conducting any searches:

• Formulate the research question using the PICO framework (Population, Intervention, Comparator, Outcome)
• Establish inclusion and exclusion criteria with explicit justification
• Define the search strategy: databases, date range, language restrictions
• Specify outcome measures and effect size metrics
• Pre-register the protocol (PROSPERO or OSF recommended)
• Document any planned sensitivity or subgroup analyses

Step 2: Multi-Database Systematic Search

Execute comprehensive searches across multiple bibliographic databases:

• PubMed/MEDLINE: Biomedical and clinical literature via structured MeSH queries
• arXiv: Preprints in quantitative and computational fields
• Semantic Scholar: AI-augmented citation graph and full-text search
• CrossRef: DOI-based metadata and cross-publisher discovery
• Construct database-specific search strings from the master strategy
• Document exact queries, dates, result counts; deduplicate exports
• Supplement with citation chaining (forward and backward) on key papers

Step 3: Screening and Eligibility Assessment

Apply a two-stage screening process to identify eligible studies:

• Title/abstract screening: Apply inclusion criteria, flag uncertain cases
• Full-text assessment: Evaluate against all criteria, document exclusion reasons
• Track inter-rater agreement (Cohen's kappa) if multiple reviewers
• Maintain a log of all screening decisions for the PRISMA flow diagram
• Resolve disagreements through discussion or third-reviewer arbitration

Step 4: Structured Data Extraction

Extract pre-defined data elements from each included study:

• Study characteristics: design, setting, sample size, follow-up duration
• Population, intervention, comparator: demographics, dosage, duration
• Outcomes and results: endpoints, effect estimates, confidence intervals
• Quality indicators: randomization method, blinding, attrition, funding

Use ScienceClaw information extraction to assist with structured data capture from PDF full texts, reducing manual effort and transcription errors.

Step 5: Risk of Bias and Quality Assessment

Evaluate methodological quality of each included study:

• Apply appropriate tools (RoB 2 for RCTs, ROBINS-I for non-randomized, Newcastle-Ottawa)
• Assess each domain: selection, performance, detection, attrition, reporting
• Generate risk-of-bias summary figures (traffic light plots)
• Evaluate overall certainty of evidence using GRADE framework
• Document judgments with supporting quotations from study texts

Step 6: Meta-Analysis and Evidence Synthesis

Perform quantitative synthesis when studies are sufficiently homogeneous:

• Calculate standardized effect sizes (SMD, OR, RR, HR as appropriate)
• Fit random-effects or fixed-effects meta-analysis models
• Generate forest plots with study-level and pooled estimates
• Assess heterogeneity: I-squared statistic, Cochran's Q test, tau-squared
• Subgroup and sensitivity analyses: leave-one-out, trim-and-fill, funnel plots

Step 7: PRISMA Reporting and Final Output

Compile the review following PRISMA 2020 reporting standards:

• PRISMA flow diagram with identification, screening, eligibility, inclusion counts
• Characteristics of included studies table
• Risk-of-bias summary and individual study assessments
• Forest plots, funnel plots, and subgroup analysis figures
• Summary of findings table with GRADE certainty ratings
• Complete PRISMA 2020 checklist (Page et al., BMJ 2021;372:n71) cross-referenced to report sections