Experimentation Analytics
rampstackco/claude-skills
How to read experiment results without fooling yourself. An agent skill from rampstackco/claude-skills.
Run hypothesis tests, analyze A/B experiment results, calculate sample sizes, and interpret statistical significance with effect sizes.
$ npx skills add alirezarezvani/claude-skills --skill statistical-analyst -a claude-codeProject install by default; add -g for ~/.claude/skills/.
$ gh skill install alirezarezvani/claude-skills statistical-analyst --agent claude-codeProject scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).
$ git clone --depth 1 https://github.com/alirezarezvani/claude-skills.git skills-src && mkdir -p .claude/skills && cp -r skills-src/engineering/statistical-analyst/skills/statistical-analyst .claude/skills/statistical-analyst && rm -rf skills-srcUse ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.
Claude Code skills documentation · loads skills from .claude/skills/
Install the "statistical-analyst" agent skill from https://github.com/alirezarezvani/claude-skills/tree/main/engineering/statistical-analyst/skills/statistical-analyst into .claude/skills/statistical-analyst/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "statistical-analyst", then confirm the skill loads.Claude Code copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$skill-installer install https://github.com/alirezarezvani/claude-skills/tree/main/engineering/statistical-analyst/skills/statistical-analystType this inside Codex. $skill-installer <name> installs a curated skill from openai/skills. The installer writes to $CODEX_HOME/skills (default ~/.codex/skills). Restart Codex if the skill does not show up.
$ npx skills add alirezarezvani/claude-skills --skill statistical-analyst -a codexProject install goes to .agents/skills/; add -g for ~/.codex/skills/.
$ gh skill install alirezarezvani/claude-skills statistical-analyst --agent codexProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/alirezarezvani/claude-skills.git skills-src && mkdir -p .agents/skills && cp -r skills-src/engineering/statistical-analyst/skills/statistical-analyst .agents/skills/statistical-analyst && rm -rf skills-srcUse ~/.agents/skills/ instead of .agents/skills for a personal install.
Codex skills documentation · loads skills from .agents/skills/
Install the "statistical-analyst" agent skill from https://github.com/alirezarezvani/claude-skills/tree/main/engineering/statistical-analyst/skills/statistical-analyst into .agents/skills/statistical-analyst/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "statistical-analyst", then confirm the skill loads.Codex copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add alirezarezvani/claude-skills --skill statistical-analyst -a cursorProject install goes to .agents/skills/; add -g for ~/.cursor/skills/.
$ gh skill install alirezarezvani/claude-skills statistical-analyst --agent cursorProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/alirezarezvani/claude-skills.git skills-src && mkdir -p .cursor/skills && cp -r skills-src/engineering/statistical-analyst/skills/statistical-analyst .cursor/skills/statistical-analyst && rm -rf skills-srcUse ~/.cursor/skills/ instead of .cursor/skills for a personal install.
Cursor skills documentation · loads skills from .cursor/skills/, .agents/skills/, .claude/skills/, .codex/skills/
Install the "statistical-analyst" agent skill from https://github.com/alirezarezvani/claude-skills/tree/main/engineering/statistical-analyst/skills/statistical-analyst into .cursor/skills/statistical-analyst/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "statistical-analyst", then confirm the skill loads.Cursor copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gemini skills install https://github.com/alirezarezvani/claude-skills.git --path engineering/statistical-analyst/skills/statistical-analyst--scope user (default) or --scope workspace; --path is the subfolder of the repo that holds the skill; --consent skips the security confirmation prompt.
$ npx skills add alirezarezvani/claude-skills --skill statistical-analyst -a gemini-cliProject install goes to .agents/skills/; add -g for ~/.gemini/skills/.
$ gh skill install alirezarezvani/claude-skills statistical-analyst --agent gemini-cliProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/alirezarezvani/claude-skills.git skills-src && mkdir -p .gemini/skills && cp -r skills-src/engineering/statistical-analyst/skills/statistical-analyst .gemini/skills/statistical-analyst && rm -rf skills-srcUse ~/.gemini/skills/ instead of .gemini/skills for a personal install, then run /skills reload.
Gemini CLI skills documentation · loads skills from .gemini/skills/, .agents/skills/
Install the "statistical-analyst" agent skill from https://github.com/alirezarezvani/claude-skills/tree/main/engineering/statistical-analyst/skills/statistical-analyst into .gemini/skills/statistical-analyst/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "statistical-analyst", then confirm the skill loads.Gemini CLI copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gh skill install alirezarezvani/claude-skills statistical-analystInstalls for Copilot at project scope by default; add --scope user for a personal install. Preview a skill first with gh skill preview. Needs GitHub CLI 2.90.0 or later (public preview).
$ npx skills add alirezarezvani/claude-skills --skill statistical-analyst -a github-copilotProject install goes to .agents/skills/; add -g for ~/.copilot/skills/.
$ git clone --depth 1 https://github.com/alirezarezvani/claude-skills.git skills-src && mkdir -p .github/skills && cp -r skills-src/engineering/statistical-analyst/skills/statistical-analyst .github/skills/statistical-analyst && rm -rf skills-srcUse ~/.copilot/skills/ instead of .github/skills for a personal install. Commit .github/skills so cloud agent and code review can use it.
GitHub Copilot skills documentation · loads skills from .github/skills/, .claude/skills/, .agents/skills/
Install the "statistical-analyst" agent skill from https://github.com/alirezarezvani/claude-skills/tree/main/engineering/statistical-analyst/skills/statistical-analyst into .github/skills/statistical-analyst/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "statistical-analyst", then confirm the skill loads.GitHub Copilot copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add alirezarezvani/claude-skills --skill statistical-analyst -a opencodeOpenCode documents no install command of its own. Project install goes to .agents/skills/; add -g for ~/.config/opencode/skills/.
$ gh skill install alirezarezvani/claude-skills statistical-analyst --agent opencodeProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/alirezarezvani/claude-skills.git skills-src && mkdir -p .opencode/skills && cp -r skills-src/engineering/statistical-analyst/skills/statistical-analyst .opencode/skills/statistical-analyst && rm -rf skills-srcUse ~/.config/opencode/skills/ instead of .opencode/skills for a personal install.
OpenCode skills documentation · loads skills from .opencode/skills/, .claude/skills/, .agents/skills/
Install the "statistical-analyst" agent skill from https://github.com/alirezarezvani/claude-skills/tree/main/engineering/statistical-analyst/skills/statistical-analyst into .opencode/skills/statistical-analyst/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "statistical-analyst", then confirm the skill loads.OpenCode copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
statistical-analystRun hypothesis tests, analyze A/B experiment results, calculate sample sizes, and interpret statistical significance with effect sizes.
Statistical Analyst is an agent skill from alirezarezvani/claude-skills. Run hypothesis tests, analyze A/B experiment results, calculate sample sizes, and interpret statistical significance with effect sizes. Use when you need to validate whether observed differences are real, size an experiment correctly before launch, or interpret test results with confidence.
Its SKILL.md is about 2.5k tokens, which your agent loads only when the skill is triggered. The skill folder holds 6 other files, including scripts and reference files (for example `references/statistical-testing-concepts.md`, `scripts/confidence_interval.py` and `scripts/hypothesis_tester.py`).
It sits in Data & Analytics, covering Statistics, Experimental design and A/B testing. The repository describes itself as: 380 Claude Code skills & agent skills & plugins (30+ Agents, 70+ custom commands, 380+ skills, customizable references, scripts)for Claude Code, Codex, Gemini CLI, Cursor, and 8… The licence is MIT.
5 steps, taken from the first numbered list in SKILL.md.
Read from SKILL.md and the folder at commit 19392f7. It shows what the files ask for, not the result of running them.
Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.
From allowed-tools in the SKILL.md frontmatter.
Ships 3 files in scripts/ (Python), which the agent can run.
Shell commands in SKILL.md call:
python3From the folder's file list and the shell code blocks in SKILL.md.
No URLs in SKILL.md.
From URLs in SKILL.md, links to its own repository left out.
Names no API keys, tokens, secrets or passwords.
From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.
Statistical Analyst loads about 2.5k tokens when it runs, and up to ~4.1k if it reads all its reference files. Until then it costs about 78 tokens; SKILL.md has 952 words of instructions outside code blocks.
Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.
The automated check found no risky patterns in SKILL.md.
Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); the scripts in this folder are not scanned.
The full file from alirezarezvani/claude-skills at commit 19392f7, republished under its MIT licence (© alirezarezvani). 952 words, ~2,495 tokens.
.claude/skills/statistical-analyst/SKILL.md (or your agent's skills folder). This skill also uses 4 other files; get the full folder from GitHub.You are an expert statistician and data scientist. Your goal is to help teams make decisions grounded in statistical evidence — not gut feel. You distinguish signal from noise, size experiments correctly before they start, and interpret results with full context: significance, effect size, power, and practical impact.
You treat "statistically significant" and "practically significant" as separate questions and always answer both.
Use when an experiment has already run and you have result data.
hypothesis_tester.py with appropriate methodUse before launching a test to ensure it will be conclusive.
sample_size_calculator.py to get required N per variantUse when someone shares a result and asks "is this significant?" or "what does this mean?"
scripts/hypothesis_tester.pyRun Z-test (proportions), two-sample t-test (means), or Chi-square test (categorical). Returns p-value, confidence interval, effect size, and a plain-English verdict.
# Z-test for two proportions (A/B conversion rates)
python3 scripts/hypothesis_tester.py --test ztest \
--control-n 5000 --control-x 250 \
--treatment-n 5000 --treatment-x 310
# Two-sample t-test (comparing means, e.g. revenue per user)
python3 scripts/hypothesis_tester.py --test ttest \
--control-mean 42.3 --control-std 18.1 --control-n 800 \
--treatment-mean 46.1 --treatment-std 19.4 --treatment-n 820
# Chi-square test (multi-category outcomes)
python3 scripts/hypothesis_tester.py --test chi2 \
--observed "120,80,50" --expected "100,100,50"
# Output JSON for downstream use
python3 scripts/hypothesis_tester.py --test ztest \
--control-n 5000 --control-x 250 \
--treatment-n 5000 --treatment-x 310 \
--format jsonscripts/sample_size_calculator.pyCalculate required sample size per variant before launching an experiment.
# Proportion test (conversion rate experiment)
python3 scripts/sample_size_calculator.py --test proportion \
--baseline 0.05 --mde 0.20 --alpha 0.05 --power 0.80
# Mean test (continuous metric experiment)
python3 scripts/sample_size_calculator.py --test mean \
--baseline-mean 42.3 --baseline-std 18.1 --mde 0.10 \
--alpha 0.05 --power 0.80
# Show tradeoff table across power levels
python3 scripts/sample_size_calculator.py --test proportion \
--baseline 0.05 --mde 0.20 --table
# Output JSON
python3 scripts/sample_size_calculator.py --test proportion \
--baseline 0.05 --mde 0.20 --format jsonscripts/confidence_interval.pyCompute confidence intervals for a proportion or mean. Use for reporting observed metrics with uncertainty bounds.
# CI for a proportion
python3 scripts/confidence_interval.py --type proportion \
--n 1200 --x 96
# CI for a mean
python3 scripts/confidence_interval.py --type mean \
--n 800 --mean 42.3 --std 18.1
# Custom confidence level
python3 scripts/confidence_interval.py --type proportion \
--n 1200 --x 96 --confidence 0.99
# Output JSON
python3 scripts/confidence_interval.py --type proportion \
--n 1200 --x 96 --format json| Scenario | Metric | Test |
|---|---|---|
| A/B conversion rate (clicked/not) | Proportion | Z-test for two proportions |
| A/B revenue, load time, session length | Continuous mean | Two-sample t-test (Welch's) |
| A/B/C/n multi-variant with categories | Categorical counts | Chi-square |
| Single sample vs. known value | Mean vs. constant | One-sample t-test |
| Non-normal data, small n | Rank-based | Use Mann-Whitney U (flag for human) |
When NOT to use these tools:
Use this after running the test:
| p-value | Effect Size | Practical Impact | Decision |
|---|---|---|---|
| < α | Large / Medium | Meaningful | ✅ Ship |
| < α | Small | Negligible | ⚠️ Hold — statistically significant but not worth the complexity |
| ≥ α | — | — | 🔁 Extend (if underpowered) or ❌ Kill |
| < α | Any | Negative UX | ❌ Kill regardless |
Always ask: "If this effect were exactly as measured, would the business care?" If no — don't ship on significance alone.
Effect sizes translate statistical results into practical language:
Cohen's d (means):
| d | Interpretation |
|---|---|
| < 0.2 | Negligible |
| 0.2–0.5 | Small |
| 0.5–0.8 | Medium |
| > 0.8 | Large |
Cohen's h (proportions):
| h | Interpretation |
|---|---|
| < 0.2 | Negligible |
| 0.2–0.5 | Small |
| 0.5–0.8 | Medium |
| > 0.8 | Large |
Cramér's V (chi-square):
| V | Interpretation |
|---|---|
| < 0.1 | Negligible |
| 0.1–0.3 | Small |
| 0.3–0.5 | Medium |
| > 0.5 | Large |
Surface these unprompted when you spot the signals:
| Request | Deliverable |
|---|---|
| "Did our test win?" | Significance report: p-value, CI, effect size, verdict, caveats |
| "How big should our test be?" | Sample size report with power/MDE tradeoff table |
| "What's the confidence interval for X?" | CI report with margin of error and interpretation |
| "Is this difference real?" | Hypothesis test with plain-English conclusion |
| "How long should we run this?" | Duration estimate = (required N per variant) / (daily traffic per variant) |
| "We tested 5 things — what's significant?" | Multiple comparison analysis with Bonferroni-adjusted thresholds |
Tag every finding with confidence:
Structure all results as:
Bottom Line — One sentence: "Treatment increased conversion by 1.2pp (95% CI: 0.4–2.0pp). Result is statistically significant (p=0.003) with a small effect (h=0.18). Recommend shipping."
What — The numbers: observed rates/means, difference, p-value, CI, effect size
Why It Matters — Business translation: what does the effect size mean in revenue, users, or decisions?
How to Act — Ship / hold / extend / kill with specific rationale
| Skill | Use When |
|---|---|
marketing-skill/ab-test-setup | Designing the experiment before it runs — randomization, instrumentation, holdout |
engineering/data-quality-auditor | Verifying input data integrity before running any statistical test |
product-team/experiment-designer | Structuring the hypothesis, success metrics, and guardrail metrics |
product-team/product-analytics | Analyzing product funnel and retention metrics |
finance/saas-metrics-coach | Interpreting SaaS KPIs that may feed into experiments (ARR, churn, LTV) |
marketing-skill/campaign-analytics | Statistical analysis of marketing campaign performance |
When NOT to use this skill:
marketing-skill/ab-test-setup or product-team/experiment-designerengineering/data-quality-auditor firstreferences/statistical-testing-concepts.md — t-test, Z-test, chi-square theory; p-value interpretation; Type I/II errors; power analysis math© alirezarezvani, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file
SKILL.md and 4 other files (scripts, references) in engineering/statistical-analyst/skills/statistical-analyst of alirezarezvani/claude-skills.
Open the folder on GitHubat commit 19392f7
We found 1 copy of this SKILL.md (exact, near-identical or edited) in other folders, from 1 other GitHub owner. This page covers the copy in alirezarezvani/claude-skills, which our catalogue first saw on October 7, 2026.
Statistical Analyst next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.
| Skill | Stars | Used in | Tokens | Auto-check | Licence | Repo updated |
|---|---|---|---|---|---|---|
| Statistical Analyst this skillalirezarezvani/claude-skills | 28k | 1 repos | ~2.5k | Automated safety check: Pass | MIT | |
| Experimentation Analyticsrampstackco/claude-skills | 935 | 1 repos | ~8.9k | Automated safety check: Pass | MIT | |
| Power Analysisgaasher/Agent-Loop-Skills | 174 | — | ~2.2k | Automated safety check: Pass | MIT | |
| Data Scientistmagnus919/hermes-profiles | 278 | — | ~3.3k | Automated safety check: Pass | MIT | |
| Data Scientistmagnus919/agent-skills | 111 | — | ~4.1k | Automated safety check: Pass | MIT | |
| Review Experiment Resultsharness/harness-skills | 115 | — | ~3.9k | Automated safety check: Pass | Apache-2.0 |
rampstackco/claude-skills
How to read experiment results without fooling yourself. An agent skill from rampstackco/claude-skills.
gaasher/Agent-Loop-Skills
A skill your agent uses when the user is planning a two-arm comparison (an A/B test, a simple RCT, a behavioral study, or a two-model/two-config evaluation) and needs to size it and preregister it…
magnus919/hermes-profiles
PhD-level expertise in data science, statistics, and machine learning.
magnus919/agent-skills
A skill your agent uses for PhD-level expertise in data science, statistics, and machine learning: rigorous statistical analysis, experimental design, causal inference, advanced modeling, research…
harness/harness-skills
Explain Harness FME experiment results: winner determination, statistical significance, guardrail metric impact, and data-quality caveats (low sample size, missing data).
ai-analyst-lab/ai-analyst
The analysis and lifecycle owner for experiments. An agent skill from ai-analyst-lab/ai-analyst.
alirezarezvani/claude-skills
Writes INVEST-checked user stories with acceptance criteria, splits epics, plans sprints from velocity and ranks the backlog with a weighted score.
alirezarezvani/claude-skills
OKR cascade toolkit for product leaders: generates aligned company-to-team OKRs from five strategy types and scores how well they line up.
alirezarezvani/claude-skills
App Store Optimization (ASO) toolkit for researching keywords, analyzing competitor rankings, generating metadata suggestions, and improving app visibility on Apple App Store and Google Play Store.
alirezarezvani/claude-skills
Design AWS architectures for startups using serverless patterns and IaC templates.
alirezarezvani/claude-skills
Calculates attribution, funnel and ROI figures for marketing campaigns with three Python scripts that need only the standard library.
alirezarezvani/claude-skills
Reverse-engineers a frontend, backend or fullstack codebase into a product requirements document with per-page docs, an enum dictionary and an API inventory.
Run hypothesis tests, analyze A/B experiment results, calculate sample sizes, and interpret statistical significance with effect sizes. Statistical Analyst is an agent skill from alirezarezvani/claude-skills. Run hypothesis tests, analyze A/B experiment results, calculate sample sizes, and interpret statistical significance with effect sizes.
Statistical Analyst fits situations like: you need to validate whether observed differences are real; size an experiment correctly before launch; interpret test results with confidence.
Run `npx skills add alirezarezvani/claude-skills --skill statistical-analyst -a claude-code`. Or copy the skill folder (engineering/statistical-analyst/skills/statistical-analyst in alirezarezvani/claude-skills) into .claude/skills/statistical-analyst in your project. Claude Code loads it when a task matches its description.
Run `npx skills add alirezarezvani/claude-skills --skill statistical-analyst -a codex`. Or copy the skill folder (engineering/statistical-analyst/skills/statistical-analyst in alirezarezvani/claude-skills) into .agents/skills/statistical-analyst in your project. Codex loads it when a task matches its description.
Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add alirezarezvani/claude-skills --skill statistical-analyst -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/statistical-analyst, .gemini/skills/statistical-analyst, .github/skills/statistical-analyst and .opencode/skills/statistical-analyst in your project.
Going by SKILL.md and its folder, Statistical Analyst needs Python for the scripts in its folder and the command-line tools its instructions call (python3). Our summary lists: Python 3.
SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.
Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. The check reads SKILL.md only: the scripts in the folder are not scanned, so read them before running anything.
Statistical Analyst is published under the MIT licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.
About 2.5k tokens (SKILL.md is roughly 10k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full. Its references folder adds about 1.6k tokens, read only when the agent opens those files.
Skills that share tags, products or a category with Statistical Analyst: Experimentation Analytics (rampstackco/claude-skills, 935 stars), Power Analysis (gaasher/Agent-Loop-Skills, 174 stars), Data Scientist (magnus919/hermes-profiles, 278 stars) and Data Scientist (magnus919/agent-skills, 111 stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.
alirezarezvani (a GitHub user) maintains it in alirezarezvani/claude-skills, which has 27,788 GitHub stars. The repository holds 342 skills in this directory. The repository was last updated on August 30, 2026.
Source: alirezarezvani/claude-skills on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.