Ab Test Analysis
nimrodfisher/data-analytics-skills
Rigorous A/B test statistical analysis. An agent skill from nimrodfisher/data-analytics-skills.
Design and run product experiments at a practical PM level — A/B tests, hypothesis tests, rollouts, feature flags, and reading results without pretending to be a statistician.
Install Experimentation
$ npx skills add andreaskelm/pm-brain --skill experimentation -a claude-codeProject install by default; add -g for ~/.claude/skills/.
$ gh skill install andreaskelm/pm-brain experimentation --agent claude-codeProject scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).
$ git clone --depth 1 https://github.com/andreaskelm/pm-brain.git skills-src && mkdir -p .claude/skills && cp -r skills-src/.claude/skills/experimentation .claude/skills/experimentation && rm -rf skills-srcUse ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.
Claude Code skills documentation · loads skills from .claude/skills/
Install the "experimentation" agent skill from https://github.com/andreaskelm/pm-brain/tree/main/.claude/skills/experimentation into .claude/skills/experimentation/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "experimentation", then confirm the skill loads.Claude Code copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$skill-installer install https://github.com/andreaskelm/pm-brain/tree/main/.claude/skills/experimentationType this inside Codex. $skill-installer <name> installs a curated skill from openai/skills. The installer writes to $CODEX_HOME/skills (default ~/.codex/skills). Restart Codex if the skill does not show up.
$ npx skills add andreaskelm/pm-brain --skill experimentation -a codexProject install goes to .agents/skills/; add -g for ~/.codex/skills/.
$ gh skill install andreaskelm/pm-brain experimentation --agent codexProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/andreaskelm/pm-brain.git skills-src && mkdir -p .agents/skills && cp -r skills-src/.claude/skills/experimentation .agents/skills/experimentation && rm -rf skills-srcUse ~/.agents/skills/ instead of .agents/skills for a personal install.
Codex skills documentation · loads skills from .agents/skills/
Install the "experimentation" agent skill from https://github.com/andreaskelm/pm-brain/tree/main/.claude/skills/experimentation into .agents/skills/experimentation/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "experimentation", then confirm the skill loads.Codex copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add andreaskelm/pm-brain --skill experimentation -a cursorProject install goes to .agents/skills/; add -g for ~/.cursor/skills/.
$ gh skill install andreaskelm/pm-brain experimentation --agent cursorProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/andreaskelm/pm-brain.git skills-src && mkdir -p .cursor/skills && cp -r skills-src/.claude/skills/experimentation .cursor/skills/experimentation && rm -rf skills-srcUse ~/.cursor/skills/ instead of .cursor/skills for a personal install.
Cursor skills documentation · loads skills from .cursor/skills/, .agents/skills/, .claude/skills/, .codex/skills/
Install the "experimentation" agent skill from https://github.com/andreaskelm/pm-brain/tree/main/.claude/skills/experimentation into .cursor/skills/experimentation/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "experimentation", then confirm the skill loads.Cursor copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gemini skills install https://github.com/andreaskelm/pm-brain.git --path .claude/skills/experimentation--scope user (default) or --scope workspace; --path is the subfolder of the repo that holds the skill; --consent skips the security confirmation prompt.
$ npx skills add andreaskelm/pm-brain --skill experimentation -a gemini-cliProject install goes to .agents/skills/; add -g for ~/.gemini/skills/.
$ gh skill install andreaskelm/pm-brain experimentation --agent gemini-cliProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/andreaskelm/pm-brain.git skills-src && mkdir -p .gemini/skills && cp -r skills-src/.claude/skills/experimentation .gemini/skills/experimentation && rm -rf skills-srcUse ~/.gemini/skills/ instead of .gemini/skills for a personal install, then run /skills reload.
Gemini CLI skills documentation · loads skills from .gemini/skills/, .agents/skills/
Install the "experimentation" agent skill from https://github.com/andreaskelm/pm-brain/tree/main/.claude/skills/experimentation into .gemini/skills/experimentation/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "experimentation", then confirm the skill loads.Gemini CLI copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gh skill install andreaskelm/pm-brain experimentationInstalls for Copilot at project scope by default; add --scope user for a personal install. Preview a skill first with gh skill preview. Needs GitHub CLI 2.90.0 or later (public preview).
$ npx skills add andreaskelm/pm-brain --skill experimentation -a github-copilotProject install goes to .agents/skills/; add -g for ~/.copilot/skills/.
$ git clone --depth 1 https://github.com/andreaskelm/pm-brain.git skills-src && mkdir -p .github/skills && cp -r skills-src/.claude/skills/experimentation .github/skills/experimentation && rm -rf skills-srcUse ~/.copilot/skills/ instead of .github/skills for a personal install. Commit .github/skills so cloud agent and code review can use it.
GitHub Copilot skills documentation · loads skills from .github/skills/, .claude/skills/, .agents/skills/
Install the "experimentation" agent skill from https://github.com/andreaskelm/pm-brain/tree/main/.claude/skills/experimentation into .github/skills/experimentation/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "experimentation", then confirm the skill loads.GitHub Copilot copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add andreaskelm/pm-brain --skill experimentation -a opencodeOpenCode documents no install command of its own. Project install goes to .agents/skills/; add -g for ~/.config/opencode/skills/.
$ gh skill install andreaskelm/pm-brain experimentation --agent opencodeProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/andreaskelm/pm-brain.git skills-src && mkdir -p .opencode/skills && cp -r skills-src/.claude/skills/experimentation .opencode/skills/experimentation && rm -rf skills-srcUse ~/.config/opencode/skills/ instead of .opencode/skills for a personal install.
OpenCode skills documentation · loads skills from .opencode/skills/, .claude/skills/, .agents/skills/
Install the "experimentation" agent skill from https://github.com/andreaskelm/pm-brain/tree/main/.claude/skills/experimentation into .opencode/skills/experimentation/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "experimentation", then confirm the skill loads.OpenCode copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
Facts
- Skill name
experimentation- GitHub stars
- 234
- Token cost
- ~2.3k tokens
- SKILL.md length
- 1,309 words
- Files
- 3 (incl. references)
- Skills in repo
- 18
- Repo updated
- First seen
- Licence
- Custom licence
At a glance
Design and run product experiments at a practical PM level — A/B tests, hypothesis tests, rollouts, feature flags, and reading results without pretending to be a statistician.
- Works in 6 steps: Preflight (before any design doc) → Lock the hypothesis and the primary metric → Sample, duration, and practical power → …
- The user says A/B test
- SKILL.md covers Is this the right thing?, Step 1 — Preflight (before any…, Step 2 — Lock the hypothesis… and Step 3 — Sample, duration, and…, plus 7 more sections
- Instructions only: no scripts, shell commands, URLs or credentials in SKILL.md
What it does
Experimentation is an agent skill from andreaskelm/pm-brain. Design and run product experiments at a practical PM level — A/B tests, hypothesis tests, rollouts, feature flags, and reading results without pretending to be a statistician. Covers hypothesis, primary metric, sample and duration, guardrails, and ship/kill/pivot calls. Use when the user says "A/B test", "experiment design", "hypothesis test", "rollout plan", "feature flag", "statistical significance", "how long should we run this", "can we ship the variant", or needs to decide whether a change actually worked.
Its SKILL.md is about 2.3k tokens, which your agent loads only when the skill is triggered. The skill folder holds 3 other files, including reference files (for example `references/criteria.md` and `references/design.md`).
It sits in Marketing & SEO, covering A/B testing, Statistics and Feature launches and release readiness. The repository describes itself as: AI-powered product management thinking & operating system. Playbooks, guides, templates, and frameworks that bridge PM theory to daily execution.
When your agent uses it
- The user says A/B test
- Experiment design
- Hypothesis test
- Statistical significance
Example prompts
- “A/B test”
- “experiment design”
- “hypothesis test”
- “/experimentation”
Workflow steps
6 steps, taken from the step headings in SKILL.md.
- Preflight (before any design doc)
- Lock the hypothesis and the primary metric
- Sample, duration, and practical power
- Guardrails and secondary metrics
- Rollout, feature flags, and operational hygiene
- Ship, kill, or pivot (before results arrive)
What it can do on your machine
Read from SKILL.md and the folder at commit 38696ac. It shows what the files ask for, not the result of running them.
Tool permissions
Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.
From allowed-tools in the SKILL.md frontmatter.
Runs code
No scripts in the folder and no shell commands in SKILL.md.
From the folder's file list and the shell code blocks in SKILL.md.
Network
No URLs in SKILL.md.
From URLs in SKILL.md, links to its own repository left out.
Credentials
Names no API keys, tokens, secrets or passwords.
From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.
Context cost
Experimentation loads about 2.3k tokens when it runs, and up to ~3.7k if it reads all its reference files. Until then it costs about 133 tokens; SKILL.md has 1,309 words of instructions outside code blocks.
Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.
Safety
Auto-check passed
The automated check found no risky patterns in SKILL.md.
Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.
SKILL.md
Its licence (Custom licence) doesn't allow us to republish the file, so here is its outline and opening line. It has 1,309 words (~2,334 tokens).
“An experiment is how you stop arguing about opinions and start arguing about evidence — but only if you decided what would convince you before the numbers land. The failure mode I see most: someone runs a test because "we…”
- name
- experimentation
Files
SKILL.md and 2 other files (references) in .claude/skills/experimentation of andreaskelm/pm-brain.
- SKILL.md
- references/criteria.md
- references/design.md
Open the folder on GitHubat commit 38696ac
Compare with similar skills
Experimentation next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.
| Skill | Stars | Used in | Tokens | Auto-check | Licence | Repo updated |
|---|---|---|---|---|---|---|
| Experimentation this skillandreaskelm/pm-brain | 234 | — | ~2.3k | Automated safety check: Pass | Custom licence | |
| Ab Test Analysisnimrodfisher/data-analytics-skills | 470 | — | ~708 | Automated safety check: Pass | MIT | |
| Measure Experiment Resultsproduct-on-purpose/pm-skills | 716 | — | ~989 | Automated safety check: Pass | Apache-2.0 | |
| Experimentai-analyst-lab/ai-analyst | 304 | — | ~2k | Automated safety check: Pass | MIT | |
| Mkt Experimentevolution-foundation/evo-nexus | 545 | — | ~1.2k | Automated safety check: Pass | Custom licence | |
| A/B Test Analysisphuryn/pm-skills | 27k | — | ~893 | Automated safety check: Pass | MIT |
Similar skills
Measure Experiment Results
product-on-purpose/pm-skills
Documents the results of a completed experiment or A/B test with statistical analysis, learnings, and recommendations.
Experiment
ai-analyst-lab/ai-analyst
The analysis and lifecycle owner for experiments. An agent skill from ai-analyst-lab/ai-analyst.
Mkt Experiment
evolution-foundation/evo-nexus
Autonomous growth experimentation framework. An agent skill from evolution-foundation/evo-nexus.
A/B Test Analysis
phuryn/pm-skills
Validates an experiment's setup, works out lift, p-value and confidence interval from A/B test data, and recommends whether to ship, extend or stop.
Statistical Analyst
alirezarezvani/claude-skills
Run hypothesis tests, analyze A/B experiment results, calculate sample sizes, and interpret statistical significance with effect sizes.
More from andreaskelm/pm-brain
All 18 skills in this repoAI Product Management
andreaskelm/pm-brain
Ship and spec AI features, LLM products, agents, copilots, and generative UX — including when to use a model vs.
Discovery Synthesis
andreaskelm/pm-brain
Plan customer discovery, turn interview snapshots into synthesis and evidence-based opportunities, build or update an Opportunity Solution Tree, map jobs and segments, and design RAT tests for the…
Eng Design Collab
andreaskelm/pm-brain
Partner effectively with engineering and design — feasibility and scope negotiation, tech debt tradeoffs, design reviews, discovery with builders, and PRD handoffs that don't get thrown away.
Launch Gtm
andreaskelm/pm-brain
Plan, tighten, or review a product launch and go-to-market motion: rollout phases, beta and GA readiness, sales enablement, marketing launch, and release comms (internal and external).
North Star
andreaskelm/pm-brain
Define, sharpen, or audit a North Star metric and its input metrics tree, and decide which product metrics actually matter (leading vs.
Okr
andreaskelm/pm-brain
Write, draft, review, or fix OKRs (objectives and key results) for a team, product area, or quarter, and run weekly confidence check-ins, mid-cycle adjustments, and end-of-cycle grading.
Related
Categories
Questions about Experimentation
What does Experimentation do?
Design and run product experiments at a practical PM level — A/B tests, hypothesis tests, rollouts, feature flags, and reading results without pretending to be a statistician. Experimentation is an agent skill from andreaskelm/pm-brain. Design and run product experiments at a practical PM level — A/B tests, hypothesis tests, rollouts, feature flags, and reading results without pretending to be a statistician.
When should I use Experimentation?
Experimentation fits situations like: the user says A/B test; experiment design; hypothesis test; statistical significance.
How do I install Experimentation in Claude Code?
Run `npx skills add andreaskelm/pm-brain --skill experimentation -a claude-code`. Or copy the skill folder (.claude/skills/experimentation in andreaskelm/pm-brain) into .claude/skills/experimentation in your project. Claude Code loads it when a task matches its description.
How do I install Experimentation in Codex?
Run `npx skills add andreaskelm/pm-brain --skill experimentation -a codex`. Or copy the skill folder (.claude/skills/experimentation in andreaskelm/pm-brain) into .agents/skills/experimentation in your project. Codex loads it when a task matches its description.
Can I use Experimentation in Cursor, Gemini CLI or GitHub Copilot?
Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add andreaskelm/pm-brain --skill experimentation -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/experimentation, .gemini/skills/experimentation, .github/skills/experimentation and .opencode/skills/experimentation in your project.
What does Experimentation need to run?
SKILL.md names no scripts, command-line tools or credentials: Experimentation is instructions for the agent only.
Does Experimentation access the network?
SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.
Is Experimentation safe to install?
Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.
What licence does Experimentation use?
Experimentation has a licence file (the repository's licence) that doesn't match a standard licence. Read it on GitHub before reusing the skill.
How many tokens does Experimentation use?
About 2.3k tokens (SKILL.md is roughly 9.3k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full. Its references folder adds about 1.3k tokens, read only when the agent opens those files.
What are the alternatives to Experimentation?
Skills that share tags, products or a category with Experimentation: Ab Test Analysis (nimrodfisher/data-analytics-skills, 470 stars), Measure Experiment Results (product-on-purpose/pm-skills, 716 stars), Experiment (ai-analyst-lab/ai-analyst, 304 stars) and Mkt Experiment (evolution-foundation/evo-nexus, 545 stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.
Who maintains Experimentation?
andreaskelm (a GitHub user) maintains it in andreaskelm/pm-brain, which has 234 GitHub stars. The repository holds 18 skills in this directory. The repository was last updated on October 9, 2026.
Source: andreaskelm/pm-brain on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.