Bio Clinical Biostatistics Categorical Tests

Version Compatibility

Reference examples tested with: scipy 1.12+ (Boschloo and Barnard added in 1.7), statsmodels 0.14+, pingouin 0.5+, exact2x2 (R) 1.6+, pandas 2.1+, numpy 1.26+.

Before using code patterns, verify installed versions match. If versions differ:

• Python: pip show <package> then help(module.function) to check signatures
• R packages cited for reference (exact2x2, Exact, ratesci): use packageVersion() then ?function_name to verify parameters

If code throws ImportError, AttributeError, or TypeError, introspect the installed package and adapt the example to match the actual API rather than retrying.

Categorical Association Tests for Clinical Data

"Test association between categorical variables" -> Determine whether treatment and a categorical clinical outcome are statistically independent (or that marginal proportions agree, for paired data) using a test calibrated to the design, the sample size, and the regulatory question.

Algorithmic Taxonomy

Postdoc reading: Lydersen, Fagerland & Laake 2009 Stat Med 28:1159 ("Recommended tests for association in 2x2 tables") argues Fisher's exact should be retired from routine use in favour of Boschloo or asymptotic Pearson; Fagerland-Lydersen-Laake 2013 BMC Med Res Methodol 13:91 makes the parallel case for mid-p or asymptotic McNemar over exact conditional. Regulatory practice (FDA reviewers) is moving in this direction but Fisher and exact conditional McNemar remain entrenched in many SAPs by inertia.

Decision Tree by Experimental Scenario

Chi-Square Test (Pearson, no continuity correction)

Goal: Test whether treatment group and outcome category are independent under asymptotic Type-I control.

Approach: Construct a contingency table, verify expected cell counts, compute the Pearson chi-square statistic without Yates' continuity correction.

from scipy.stats import chi2_contingency
import pandas as pd

table = pd.crosstab(df['treatment'], df['outcome'])
chi2, p, dof, expected = chi2_contingency(table, correction=False)
if (expected < 5).any():
    print('WARNING: switch to Boschloo (2x2) or permutation chi-square (RxC)')

Cochran 1954 rule (the precise version, not the textbook caricature): "no expected cell should be <1 AND no more than 20% of cells should have expected <5." The textbook "all >=5" rule is the conservative simplification. With well-balanced 2x2 trials this rarely matters; with sparse RxC tables it materially expands the asymptotic range. The R chisq.test issues a warning under the strict Cochran rule; Python users must check manually.

Why Yates' correction is now discouraged: continuity correction was introduced to approximate the exact distribution under H0 but inflates Type-II error by ~10% (D'Agostino, Chase & Belanger 1988 Am Stat 42:198). Modern computing makes Boschloo's exact test cheap; the correct fix for sparse 2x2 is Boschloo, not chi-square + continuity.

Fisher's Exact -- and why Boschloo is usually better

Goal: Test 2x2 association with exact Type-I control.

Approach: Use Fisher's exact only when historical SAP requires it; otherwise prefer Boschloo's test.

from scipy.stats import fisher_exact, boschloo_exact

odds_ratio, p_fisher = fisher_exact(table.values, alternative='two-sided')

# Boschloo (uniformly more powerful than Fisher):
# n= controls Sobol sampling resolution for the null distribution (scipy 1.12+; default 32);
# higher = more precise p-value at higher CPU cost. NOT the sample size per arm.
result = boschloo_exact(table.values, alternative='two-sided', n=64)
p_boschloo = result.pvalue

The conditioning critique (Mehta-Senchaudhuri 2003): Fisher's exact conditions on both margins of the 2x2 table, discarding information about the marginal totals. Boschloo conditions on one margin only and treats the second as a nuisance to be maximised over -- recovering the discarded information. Power gain at n=10/arm is 16-20 percentage points for moderate effects. Boschloo uses Fisher's p-value as its test statistic, then computes the exact unconditional null distribution of that p-value -- so it is automatically at least as powerful as Fisher.

Since scipy 1.10, fisher_exact returns the sample (unconditional) odds ratio, not the conditional MLE. For the conditional MLE matching R's fisher.test, use scipy.stats.contingency.odds_ratio(table, kind='conditional').

Cochran-Mantel-Haenszel (Stratified)

Goal: Test treatment-outcome association while controlling for a stratification variable; quantify the common odds ratio across strata.

Approach: Construct per-stratum 2x2 tables, compute MH pooled OR and CMH test of H0: common-OR = 1; test homogeneity via Breslow-Day.

from statsmodels.stats.contingency_tables import StratifiedTable
import pandas as pd

tables = []
for stratum in df['site'].unique():
    stratum_data = df[df['site'] == stratum]
    t = pd.crosstab(stratum_data['treatment'], stratum_data['outcome']).values
    if t.shape == (2, 2) and t.min() > 0:
        tables.append(t)

st = StratifiedTable(tables)
print(st.test_null_odds())          # CMH H0: common OR = 1
print(st.oddsratio_pooled)          # MH pooled OR
print(st.oddsratio_pooled_confint(method='normal'))
print(st.test_equal_odds())         # Breslow-Day H0: equal stratum ORs

Per-method failure modes

CMH -- Simpson's paradox masking

• Trigger: Stratum-specific ORs reverse direction across strata while the pooled MH OR appears null or modestly different from 1.
• Mechanism: CMH pools weighted log-ORs; equal-magnitude opposite-sign ORs cancel.
• Symptom: Breslow-Day p < 0.05 with stratum ORs visually reversing.
• Fix: Report stratum-specific ORs as primary; the MH pooled estimate is not a valid summary. Move to logistic regression with treatment-by-stratum interaction.

Breslow-Day -- low-power false reassurance

• Trigger: Few strata (k<5) or sparse strata (mean cell count <10).
• Mechanism: Breslow-Day chi-square has k-1 df; with k=3 and modest heterogeneity, power can be <40%.
• Symptom: Breslow-Day p > 0.5 with stratum ORs visually heterogeneous on a forest plot.
• Fix: Always supplement with a forest plot of stratum-specific ORs. Use likelihood-ratio interaction test from logistic regression as a second check.

CMH -- ignoring randomisation stratification factors

• Trigger: Randomisation was stratified (sex, region, baseline severity) but the primary analysis pools across strata.
• Mechanism: Stratified randomisation removes between-stratum variability that the unstratified SE still counts.
• Symptom: Over-conservative inference -- SE biased upward, CIs too wide, Type-I below nominal, power loss (Kahan-Morris 2012).
• Fix: Strata variables from randomisation MUST appear in analysis -- either CMH, logistic regression with strata, or stratified log-rank.

McNemar's Test for Paired Binary Data

Goal: Test the null of marginal homogeneity (P(positive at time 1) = P(positive at time 2)) for paired binary observations.

Approach: Default to asymptotic McNemar without continuity correction when discordant pairs >=25; switch to mid-p when discordant pairs <25; reserve exact conditional only when regulator-mandated.

from statsmodels.stats.contingency_tables import mcnemar
import numpy as np

# table[i,j] = count with outcome i at time 1 and j at time 2
table = np.array([[45, 15], [5, 35]])  # b=15, c=5 discordant

# Asymptotic, no continuity correction -- the Fagerland 2013 recommended default
result = mcnemar(table, exact=False, correction=False)
print(result.statistic, result.pvalue)

# Exact conditional (Liddell 1983) -- only when b+c is very small or required by SAP
result_exact = mcnemar(table, exact=True)

Fagerland-Lydersen-Laake 2013 BMC Med Res Methodol 13:91 simulation findings: mid-p McNemar and asymptotic McNemar (no continuity correction) outperform exact conditional McNemar across small-to-moderate samples. The exact conditional is too conservative because it conditions on a discrete margin (the discordant pair count). Their title is the methodological provocation -- "The McNemar test: asymptotic and mid-p are better than exact conditional."

Suissa-Shuster 1991 Biometrics 47:361 exact unconditional uses all N pairs (not just discordant) -- uniformly more powerful than exact conditional McNemar; sample sizes 20-40% smaller for the same power. Available in R exact2x2::mcnemarExactDP. Practically essential for matched-pair non-inferiority in diagnostic device trials.

Effect Sizes for Categorical Data

Goal: Quantify association strength beyond p-values.

Approach: Phi for 2x2, Cramer's V for RxC; bias-corrected variants in pingouin.

import numpy as np
import pingouin as pg

n = table.values.sum()
phi = np.sqrt(chi2 / n)
k = min(table.shape) - 1
cramers_v = np.sqrt(chi2 / (n * k))

# Pingouin with multiple test variants and bias correction:
expected, observed, stats = pg.chi2_independence(df, x='treatment', y='outcome')
# stats columns: test, lambda, chi2, dof, pval, cramer, power

Cohen 1988 effect-size benchmarks:

Phi equals Cramer's V for 2x2 (k=1). For RxC, only Cramer's V is valid because phi can exceed 1.

Reconciliation: When Methods Disagree