Analysis Workflow

Reproducible Analysis Modules

Use this skill for a scientific workflow with two or more analysis stages or when a stage produces scripts plus result files. It defines project organization and methods capture; load figure-style as well whenever a stage creates or revises a plot.

1. Plan module boundaries

Before writing outputs, list the modules and the dependency edges between them. Use stable ASCII names. Conventional acronyms such as QC, PCA, DEG, and GSEA may stay uppercase; otherwise prefer a short kebab-case name.

Respect a compatible layout that already exists. Do not reorganize unrelated user files merely to impose this convention.

2. Default module layout

Create only directories the module actually needs:

<module>/
├── scripts/
├── input/
├── output/
│   ├── figures/
│   └── tables/
└── README.md

• scripts/ contains the executable source for this module.
• input/ contains small module-specific inputs or a manifest/reference to the canonical data. Do not duplicate a large dataset by default.
• output/figures/ contains rendered figures from this module only.
• output/tables/ contains machine-readable results from this module only.
• README.md is the module's reproducibility record and methods source.

Shared immutable/raw data may live in project-level data/. A downstream module references an upstream output by a project-relative path; it does not silently copy or rename that output.

3. Make outputs attributable

Every output must have one producing script or recorded command. Use deterministic filenames that identify the analysis and content. Keep temporary files outside the final output directories or name them clearly as temporary.

Script persistence and process lifetime are separate concerns. Wisp's python and r runtimes retain variables and loaded objects across calls and can execute saved scripts. shell and run_in_context execute commands in fresh processes. Choose according to the user's workflow, state reuse, script requirements, and task lifecycle, using the selected environment in either case.

When an analysis depends on an expensive object already loaded in a Python or R runtime:

• keep the reproducible analysis in a project-local .py or .R file;
• execute that file with the python/r tool's script_path in the same runtime, declaring the input bindings with required_objects;
• keep heavyweight loading in a separate bootstrap script or explicit loader cell; analysis scripts consume the loaded object and must not reload it;
• use run_in_context, python file.py, or Rscript only for a deliberately fresh, state-independent batch execution.

For standalone execution, record the script path and exact command. For runtime execution, record the script path and returned source hash/runtime generation in the module README. For clean-room replay, an optional batch wrapper may load the data once and then call the same analysis functions; it is not the default hot-iteration path.

Before completing a module, verify:

1. every declared output exists and is non-empty;
2. every table can be parsed in its declared format;
3. every figure was rendered and visually inspected using figure-style;
4. README input and output paths resolve from the project root;
5. reported thresholds and parameters match the actual script.