Experiment Detail Comparator

Source: https://github.com/aipoch/medical-research-skills

Validation Shortcut

Run this minimal command first to verify the supported execution path:

python scripts/compare_methods.py --help

Experiment Detail Comparator

When to Use

Use this skill when you need to:

1. Compare two papers that apply the same or highly similar experimental method (e.g., CRISPR editing, ELISA, Western blot, synthesis protocol).
2. Identify parameter-level differences in protocols (concentration, ratio, dosage, temperature, duration, incubation, buffer composition, instrument settings).
3. Explain why protocols differ by searching additional references and extracting evidence-backed rationales.
4. Produce a structured, shareable HTML comparison report for collaborators, lab records, or reproducibility audits.
5. Standardize method extraction into machine-readable JSON for downstream analysis (QA, meta-analysis, protocol harmonization).

Key Features

• Zotero-first retrieval: locate items by title/author/DOI, then resolve PDF attachments.
• PDF → Markdown conversion via the mistral-pdf-to-markdown workflow for robust text extraction.
• Comprehensive method extraction including:
  • reagents, concentrations, ratios, volumes
  • temperatures, durations, timing
  • organisms/cell lines, sample handling, ethics/consent (when present)
  • equipment models and settings
  • experimental design (controls, replicates, randomization/blinding when stated)
  • stepwise procedure structure
• Optional table extraction from PDFs to capture dense parameter tables.
• Protocol comparison engine producing a structured diff and a human-readable comparison table.
• Literature-backed explanations: query PubMed/Europe PMC/Semantic Scholar for each major difference.
• Evidence grading (High/Medium/Low) for each explanation.
• HTML report generation with citations, highlighted differences, and graded rationale sections.

Dependencies

Runtime (Python)

• Python >=3.10

Core libraries / modules

• json (stdlib)
• subprocess (stdlib)
• pathlib (stdlib)
• re (stdlib)

Optional but recommended

• pdfplumber >=0.10.0 (PDF table extraction)
• PyPDF2 >=3.0.0 (PDF text extraction / parsing support)
• requests >=2.31.0 (download PDFs from URLs)

Install optional dependencies:

pip install "pdfplumber>=0.10.0" "PyPDF2>=3.0.0" "requests>=2.31.0"

External tools / services

• Zotero MCP tools: mcp__zotero__*
• mistral-pdf-to-markdown skill (requires MISTRAL_API_KEY)
• Literature search tools (as available in your environment):
  • PubMed
  • Europe PMC
  • Semantic Scholar
• Optional: EDirect for PubMed CLI workflows
  https://www.ncbi.nlm.nih.gov/books/NBK179288/

Environment variables (recommended)

Set in Notes/.env:

• ZOTERO_API_KEY (required for Zotero web library access; optional for local-only)
• ZOTERO_LIBRARY_TYPE (user or group)
• ZOTERO_LIBRARY_ID (your Zotero library ID)
• MISTRAL_API_KEY (required for best PDF→Markdown conversion)

Example Usage

End-to-end run (recommended)

Compare two papers by Zotero search terms:

python run_comparison.py \
  "CRISPR-Cas9 knockout in HeLa cells" \
  "Efficient genome editing in HEK293 cells" \
  ./output

Compare two papers by Zotero attachment keys:

python run_comparison.py \
  ABCDEFGHIJKLMNOPQRSTUVWXYZ123456 \
  ZYXWVUTSRQPONMLKJIHGFEDCBA654321 \
  ./output

Compare two local PDFs:

python run_comparison.py \
  ./paper1.pdf \
  ./paper2.pdf \
  ./output

Expected outputs (in ./output):

• comparison_report.html
• method_details.json
• explanations.json

Manual workflow (script-by-script)

1. Find items in Zotero and resolve PDF attachments

result1 = mcp__zotero__zotero_search_items(query="paper 1 title or author", limit=5)
item_key1 = result1[0]["key"]

result2 = mcp__zotero__zotero_search_items(query="paper 2 title or author", limit=5)
item_key2 = result2[0]["key"]

children1 = mcp__zotero__zotero_get_item_children(item_key=item_key1)
attachment_key1 = [c for c in children1 if c.get("type") == "application/pdf"][0]["key"]

children2 = mcp__zotero__zotero_get_item_children(item_key=item_key2)
attachment_key2 = [c for c in children2 if c.get("type") == "application/pdf"][0]["key"]

2. Retrieve PDFs (local cache first, download if needed)

python scripts/get_zotero_pdf.py "$ATTACHMENT_KEY1"
python scripts/get_zotero_pdf.py "$ATTACHMENT_KEY2"

3. Convert PDFs to Markdown

python scripts/convert_pdf_to_markdown.py "PATH_TO_PDF1" "temp/paper1.md"
python scripts/convert_pdf_to_markdown.py "PATH_TO_PDF2" "temp/paper2.md"

4. Extract comprehensive method details to JSON

python scripts/extract_method_section.py temp/paper1.md temp/paper1_method.json
python scripts/extract_method_section.py temp/paper2.md temp/paper2_method.json

5. Compare extracted methods

python scripts/compare_methods.py temp/paper1_method.json temp/paper2_method.json output/comparison.json

6. Search literature for explanations (per major difference)

python scripts/search_explanations.py output/comparison.json output/explanations.json

7. Generate HTML report

python scripts/generate_html_report.py \
  temp/paper1_method.json \
  temp/paper2_method.json \
  output/comparison.json \
  output/explanations.json \
  output/comparison_report.html

Implementation Details

1) Method extraction schema (what is captured)

The extraction step aims to produce a structured JSON object per paper with:

• Basic parameters

  • method name (if inferable)
  • reagents and materials
  • concentrations, ratios, dosages, volumes
  • temperatures, durations, timing
  • organisms/cell lines
  • equipment list

• Experimental design

  • control vs experimental groups
  • replicates (technical/biological; e.g., n=3)
  • sample size statements
  • randomization/blinding (only if explicitly stated)
  • block design cues (if present)

• Materials detail

  • chemicals (supplier/manufacturer, catalog/batch when present)
  • kits and manufacturers
  • media/buffers and compositions (when listed)
  • solvents

• Equipment detail

  • instrument model/manufacturer
  • instrument settings (e.g., centrifuge speed, wavelength, incubation conditions)

• Sample information

  • source and handling (fresh/frozen/fixed/dried)
  • storage conditions
  • ethics approval / consent statements (when present)

• Key steps

  • step sequence (numbered when possible)
  • step type tags (preparation / reaction / purification / detection)

2) Optional enhancements

A) Extract tables directly from PDF

Use when parameters are primarily in tables (common in materials lists, primer tables, buffer recipes):

python scripts/extract_pdf_tables.py paper.pdf output/tables.json

Output includes detected table types (e.g., materials/parameters/results) and normalized JSON rows.

B) Download and parse full PDFs (URL or local)

Use when you need full-text parsing, DOI detection, or multi-language handling:

python scripts/download_full_pdf.py https://example.com/paper.pdf ./downloads
# or
python scripts/download_full_pdf.py ./local/paper.pdf ./parsed

Features:

• automatic language detection (e.g., English/CJK/Russian)
• DOI extraction
• JSON metadata + plain text output

3) Comparison logic (protocol diff)

compare_methods(...) identifies differences across normalized fields and produces:

• a parameter comparison table (Paper 1 vs Paper 2)
• a list of significant differences (heuristics typically prioritize:
  • large numeric deltas (e.g., 2× concentration, 10°C shift)
  • categorical changes (different reagent, cell line, instrument)
  • timing/sequence changes that affect outcomes)

Example comparison table structure:

4) Literature search and explanation synthesis

For each significant difference, the skill builds targeted queries such as:

• {method_name} {parameter} optimization
• {parameter} effect on yield
• {reagent} concentration protocol comparison

Supported sources (depending on your toolchain availability):

• PubMed
• Europe PMC (including full-text term extraction when available)
• Semantic Scholar (including PDF snippet retrieval when available)

The output is an explanation object per parameter with:

• summary rationale
• citations
• extracted supporting snippets (when available)