Single Cell Analysis

Single-Cell Analysis & Reporting

Overview

This skill performs comprehensive statistical analysis on clean, integrated single-cell RNA-seq data. It transforms batch-corrected AnnData into biological insights through rigorous differential expression testing, pathway enrichment, and critical review.

Pipeline:

1. Load & Validate - Import integrated.h5ad and verify requirements met
2. Differential Expression - Pseudobulk PyDESeq2 analysis (donor x condition aggregation)
3. Pathway Enrichment - Multi-database GSEA analysis
4. Critical Review - Independent quality evaluation and synthesis
5. Final Report - Publication-ready analysis with therapeutic target recommendations

Input: processed/integrated.h5ad (from single-cell-data-prep-qc skill)

Outputs:

• DE results tables (de_*.csv)
• Pathway enrichment results (pathway_*.csv, gsea_*.csv)
• Visualization suite (volcano plots, heatmaps, pathway dotplots)
• Three reports (DRAFT, CRITICAL_REVIEW, FINAL)

Key Features:

• MANDATORY pseudobulk approach (avoids pseudoreplication)
• MANDATORY pathway enrichment (multiple databases)
• MANDATORY independent critical review
• Publication-quality visualizations throughout
• Therapeutic target prioritization

⚠️ CRITICAL: Input Requirements

This skill REQUIRES pre-processed integrated data from the data-prep-qc skill.

Required input file: workspace/{date}/{run_id}/single_cell_analyst/data/processed/integrated.h5ad

Input must contain:

• ✅ Raw counts in .X (integer counts, not normalized)
• ✅ Gene symbols as var_names (not integers)
• ✅ obs['condition'] column (disease/healthy labels)
• ✅ obs['donor_id'] column (for pseudobulk DE - preferred)
• ✅ Cell type annotations (unified_cell_type or cell_type)
• ✅ Batch-corrected embedding (X_pca_harmony for visualizations)

If you don't have integrated data yet: → First invoke the single-cell-data-prep-qc skill to prepare your data

Workspace Management

Pattern for using existing workspace:

from src.utils.workspace_manager import WorkspaceManager

# Use the SAME date/run_id from data prep skill
current_date = 'YYYY-MM-DD'  # From data prep output
current_run_id = 'XXXXXXXX'  # From data prep output (8 chars)

wm = WorkspaceManager(
    agent_name='single_cell_analyst',
    date=current_date,
    run_id=current_run_id
)

# Get paths
processed_dir = wm.get_data_path('processed')
figure_dir = wm.get_results_path('figures')
table_dir = wm.get_results_path('tables')
report_dir = wm.get_results_path('reports')

All analysis scripts and results will be added to the existing workspace.

Visualization Requirements

This skill requires publication-quality visualizations for all statistical analyses. All figures should:

• Use dpi=300 for publication quality
• Include statistical annotations (p-values, FDR)
• Use clear titles, axis labels, and legends
• Save to workspace/{date}/{run_id}/single_cell_analyst/results/figures/
• Use informative filenames (e.g., de_volcano_T_cells.png, pathway_dotplot_hallmark.png)

Required visualization categories:

1. Differential expression - Volcano plots, MA plots, heatmaps of top DE genes
2. Pathway enrichment - Dotplots, bar plots, enrichment networks
3. Target genes - Expression plots (violin, UMAP overlays) for key candidates
4. Cell type-specific analysis - Per-cell-type DE and pathway summaries

See workflow documentation for specific visualization requirements at each step.

Procedure Quick Reference

This skill contains procedures located in procedures/:

Choose DE approach based on your comparison:

• Disease vs Healthy (cross-condition) - pseudobulk_de_procedure.md - PyDESeq2 donor-level DE (requires matched dataset)
• Cell type comparisons (within disease) - within_donor_meta_analysis.md - Within-donor DE + meta-analysis (e.g., T cells vs B cells, malignant vs normal)

Always Required:

• Pathway Enrichment - pathway_enrichment_procedure.md - gseapy GSEA pathway analysis
• Critical Review - review_procedure.md - 5-category quality evaluation

⚠️ Forbidden Actions

You Must NEVER:

• Use Wilcoxon/t-test on single cells when donor_id exists (pseudoreplication)
• Skip pathway enrichment analysis
• Skip Stage 3 critical review
• Create FINAL_REPORT.md before DRAFT_REPORT.md and CRITICAL_REVIEW.md
• Proceed to next stage without validating current stage complete
• Generate figures without statistical annotations

See reference/forbidden_actions.md for full details.

Computational Resources

See reference/computational_resources.md for details.

Available resources:

• 650 GB RAM
• Up to 30-minute timeouts (1,800,000 ms)
• Multiple cores for parallel processing

Typical runtimes:

• Pseudobulk DE (per cell type): 2-5 minutes
• GSEA enrichment: 5-10 minutes per cell type
• Full analysis pipeline: 45-70 minutes

No excuses for shortcuts or skipping steps due to computational constraints.

Two-Stage Workflow

Stage 1: Statistical Analysis (DE + Pathways)

Complete workflow: See workflows/stage1_statistical_analysis.md

Objectives:

• Load and validate integrated.h5ad
• MANDATORY Checkpoint 1: Run pseudobulk differential expression
• MANDATORY Checkpoint 2: Run pathway enrichment analysis
• Generate comprehensive DE and pathway visualizations
• Create analysis summary tables

Key outputs:

• results/tables/de_*.csv - DE results per cell type
• results/tables/pathway_*.csv - Enriched pathways
• DE visualizations (volcano plots, heatmaps)
• Pathway visualizations (dotplots, bar charts)

Stage 2: Critical Review & Synthesis

Complete workflow: See workflows/stage2_review_synthesis.md

⛔ ALL SUB-STAGES ARE MANDATORY ⛔

Objectives:

• Stage 2A: Create analysis_report_DRAFT.md with all findings
• Stage 2B: Independent critical review (adopt Dr. Reviewer persona)
• Stage 2B-Action: Address review feedback (revise or fix)
• Stage 2C: Create FINAL_REPORT.md with therapeutic target recommendations

Key outputs:

• results/reports/analysis_report_DRAFT.md
• results/reports/CRITICAL_REVIEW.md
• results/reports/FINAL_REPORT.md