Power Analysis

/power-analysis — Power / MDE for study design

Compute the three interlocking quantities of an ex-ante design calculation — power, required N, and minimum detectable effect (MDE) — and emit a power section the user can paste straight into a preregistration. Analytical for standard designs; simulation-based (reusing the /simulation-study harness pattern) for non-standard ones.

Core principle: a power calculation is a design-time commitment made before the data exist. Fix any two of {effect size, N, power} and solve for the third; never back out a "power" number from a realised estimate (that is post-hoc power, and it is uninformative — see "What this skill does NOT do").

When to use

• Before launching an RCT / field / survey experiment — to choose N (or clusters) for a target MDE at 80–90% power.
• Invoked by /preregister for RCTs — the AEA RCT Registry and most IRBs require a power/MDE justification; /preregister's aea-rct style follows this skill's phases to fill that section (this skill is user-invoked only, so it is read and followed, not called).
• During R&R — when a referee asks "was this study adequately powered to detect the effect you claim?"
• Designing a Monte Carlo — to set R and sample sizes before handing off to /simulation-study.

Inputs

$ARGUMENTS may carry flags; missing pieces are elicited in Phase 0.

• --mode mde|n|power — solve for MDE given N+power, N given MDE+power, or power given N+MDE. Default mde.
• --design rct|cluster|multiarm|sim — two-arm RCT, clustered RCT (ICC), multiple arms, or simulation-based. Default inferred from the elicited design.
• --input <path> — a spec from /interview-me (under quality_reports/specs/) to pull the RQ, outcome, and design from.

Workflow

Phase 0 — Elicit the design

Gather the design parameters; ask once for anything missing rather than fabricating. Required:

• Estimand & test: primary outcome, one- vs two-sided test, alpha (default 0.05), and whether the target is a difference in means, a proportion, or a regression coefficient.
• Two of {effect size, N, power}: the effect as a raw difference and in standardized units (Cohen's d = effect / SD) — record both; power default 0.80.
• Baseline mean and SD (or baseline proportion for a binary outcome) — needed to translate raw ↔ standardized effects.
• Allocation: treated:control ratio (default 1:1; unequal allocation costs power — note it).
• Clustering: if randomization is at a group level (village, school, clinic), the ICC (ρ), the average cluster size (m), and number of clusters. Compute the design effect DEFF = 1 + (m − 1)·ρ and the effective N.
• Multiplicity: number of arms / primary outcomes; the correction (Bonferroni, Holm, or none) and whether power is per-comparison or familywise.

Echo a Pre-Flight Report (design, the two fixed quantities, the one being solved for, alpha, power, allocation, ICC/clusters, multiplicity) before computing. If the estimand or the SD source is ambiguous, stop and ask.

Phase 1 — Analytical power (standard designs)

For two-arm RCTs, clustered RCTs, and multi-arm comparisons, compute analytically. Prefer R pwr / WebPower (or a closed-form power.t.test / power.prop.test); for clustered designs inflate variance by DEFF, or use pwr on the effective N. Stata users: power twomeans / power twoproportions / power, cluster; Python: statsmodels.stats.power. Emit a short script to scripts/R/power_<slug>.R (or .do / .py) so the calc is reproducible, not a one-off console number.

• MDE mode: MDE = (z_{1−α/2} + z_{1−β}) · SE(effect), where SE is built from the SD, N, allocation, and DEFF. Report MDE in raw and standardized units.
• N mode: invert the above for total N (and #clusters when clustered) given the target MDE.
• Power mode: given N and a hypothesized effect, return achieved power.
• Multi-arm: divide alpha by the number of comparisons in the family m (Bonferroni alpha/m): m = K−1 for all-vs-control, m = K(K−1)/2 for all-pairwise. Report per-comparison and familywise power.

Sweep a grid (N or #clusters × effect size) so Phase 3 can draw a power curve and an MDE-vs-N curve.

Phase 2 — Simulation-based power (non-standard designs)

When the design is not a clean two-arm comparison — DiD / staggered event-study, IV / 2SLS, panel with serial correlation, a non-normal or censored outcome, or any estimator with no closed-form SE — switch to simulation. Reuse the /simulation-study harness exactly (see simulation-study and .claude/rules/simulation-conventions.md):

1. Seeded, parameterized DGP that embeds the hypothesized effect (and the null DGP for size). set.seed(YYYYMMDD) once; L'Ecuyer streams if parallel.
2. Estimator = the one you will actually use on the real data (e.g. fixest::feols two-way FE, did::att_gt, AER::ivreg), returning est, se, ci, p, reject.
3. Power = share of reps rejecting H0 at alpha; size = rejection rate under the null DGP (verify it is near nominal before trusting power). Report each with its Monte Carlo SE = sqrt(p(1−p)/R).
4. Sweep N (or #clusters / #periods) to trace the power curve; save the raw per-rep tibble via saveRDS() to output/.

A simulated power number without an MCSE, or without a verified size check, is not yet an answer.