Hypothesis Testing Guide

What it does

When your agent uses it

• Tasks that involve Experimental design
• Tasks that involve Statistics

Example prompts

• “/hypothesis-testing-guide”

Requirements

• Python 3

Workflow steps

7 steps, taken from the first numbered list in SKILL.md.

1. 1State hypotheses. Define H0 (null: no effect) and H1 (alternative: effect exists).
2. 2Choose significance level. Typically alpha = 0.05, but justify your choice.
3. 3Select the appropriate test. Based on data type, distribution, and design.
4. 4Check assumptions. Normality, homogeneity of variance, independence.
5. 5Compute test statistic and p-value.
6. 6Report effect size and confidence interval. p-values alone are insufficient.
7. 7Make a decision. Reject or fail to reject H0, with practical interpretation.

What it can do on your machine

Read from SKILL.md and the folder at commit bf44b3c. It shows what the files ask for, not the result of running them.

• Tool permissions

  Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

  From allowed-tools in the SKILL.md frontmatter.

• Runs code

  No scripts in the folder and no shell commands in SKILL.md (its code samples are python).

  From the folder's file list and the shell code blocks in SKILL.md.

• Network

  Links to these hosts (documentation or services it may open):

  • docs.scipy.org
  • pingouin-stats.org
  • statsmodels.org
  • apastyle.apa.org
  • machinelearningmastery.com

  From URLs in SKILL.md, links to its own repository left out.

• Credentials

  Names no API keys, tokens, secrets or passwords.

  From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Hypothesis Testing Guide loads about 2k tokens when it runs. Until then it costs about 25 tokens; SKILL.md has 526 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it

~25

When it runs · the whole SKILL.md, loaded when a task matches

~2k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from wentorai/research-plugins at commit bf44b3c, republished under its MIT licence (© wentorai). 526 words, ~1,995 tokens.

name

hypothesis-testing-guide

description

Statistical hypothesis testing, power analysis, and significance reporting

from scipy import stats
import numpy as np
import pingouin as pg

# Generate example data
control = np.random.normal(50, 10, n=30)
treatment = np.random.normal(55, 10, n=30)

# Check normality assumption
stat_c, p_c = stats.shapiro(control)
stat_t, p_t = stats.shapiro(treatment)
print(f"Normality p-values: control={p_c:.3f}, treatment={p_t:.3f}")

# Check homogeneity of variance
stat_l, p_l = stats.levene(control, treatment)
print(f"Levene's test p={p_l:.3f}")

# Run t-test
t_stat, p_val = stats.ttest_ind(control, treatment, equal_var=(p_l > 0.05))

# Effect size (Cohen's d)
cohens_d = (treatment.mean() - control.mean()) / np.sqrt(
    ((len(control)-1)*control.var() + (len(treatment)-1)*treatment.var())
    / (len(control) + len(treatment) - 2)
)

print(f"t={t_stat:.3f}, p={p_val:.4f}, Cohen's d={cohens_d:.3f}")

import pandas as pd

df = pd.DataFrame({
    'score': np.concatenate([
        np.random.normal(50, 10, 30),
        np.random.normal(55, 10, 30),
        np.random.normal(60, 10, 30)
    ]),
    'group': np.repeat(['A', 'B', 'C'], 30)
})

# ANOVA
aov = pg.anova(data=df, dv='score', between='group', detailed=True)
print(aov)

# Post-hoc pairwise comparisons (Tukey HSD)
posthoc = pg.pairwise_tukey(data=df, dv='score', between='group')
print(posthoc[['A', 'B', 'diff', 'p-tukey', 'hedges']])

# Contingency table
observed = pd.DataFrame(
    [[45, 30], [25, 50]],
    index=['Method A', 'Method B'],
    columns=['Success', 'Failure']
)

chi2, p, dof, expected = stats.chi2_contingency(observed)
cramers_v = np.sqrt(chi2 / (observed.values.sum() * (min(observed.shape) - 1)))

print(f"chi2={chi2:.3f}, p={p:.4f}, Cramer's V={cramers_v:.3f}")

from statsmodels.stats.power import TTestIndPower, FTestAnovaPower

# For a two-sample t-test
analysis = TTestIndPower()

# Calculate required sample size
n = analysis.solve_power(
    effect_size=0.5,   # Cohen's d (medium effect)
    alpha=0.05,
    power=0.80,
    ratio=1.0,         # Equal group sizes
    alternative='two-sided'
)
print(f"Required n per group: {int(np.ceil(n))}")

# Power curve
import matplotlib.pyplot as plt

sample_sizes = np.arange(10, 200, 5)
powers = [analysis.power(effect_size=0.5, nobs1=n, ratio=1.0, alpha=0.05)
          for n in sample_sizes]

fig, ax = plt.subplots()
ax.plot(sample_sizes, powers)
ax.axhline(0.8, color='red', linestyle='--', label='Power = 0.80')
ax.set_xlabel('Sample Size per Group')
ax.set_ylabel('Statistical Power')
ax.legend()
fig.savefig('power_curve.pdf')

from statsmodels.stats.multitest import multipletests

p_values = [0.01, 0.04, 0.03, 0.08, 0.002]

# Bonferroni (conservative)
reject_bonf, pvals_bonf, _, _ = multipletests(p_values, method='bonferroni')

# Benjamini-Hochberg FDR (less conservative)
reject_bh, pvals_bh, _, _ = multipletests(p_values, method='fdr_bh')

for i, p in enumerate(p_values):
    print(f"p={p:.3f} | Bonferroni: {pvals_bonf[i]:.3f} ({reject_bonf[i]}) "
          f"| BH-FDR: {pvals_bh[i]:.3f} ({reject_bh[i]})")

© wentorai, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

Just SKILL.md in skills/analysis/statistics/hypothesis-testing-guide of wentorai/research-plugins.

Open the folder on GitHubat commit bf44b3c

Used in 1 other repository

We found 1 copy of this SKILL.md (exact, near-identical or edited) in other folders, from 1 other GitHub owner. This page covers the copy in wentorai/research-plugins, which our catalogue first saw on October 7, 2026.

• brycewang-stanford/Auto-Empirical-Research-Skillsskills/43-wentorai-research-plugins/skills/analysis/statistics/hypothesis-testing-guide