Ad Test Designer
aaron-he-zhu/aaron-marketing-skills
A skill your agent uses when the user asks to "design an A/B test", "set up a creative/landing test", "run an incrementality test", or "is this result statistically and practically material?"…
Calculate A/B test statistical significance. An agent skill from guia-matthieu/clawfu-skills.
Install Ab Test Stats
$ npx skills add guia-matthieu/clawfu-skills --skill ab-test-stats -a claude-codeProject install by default; add -g for ~/.claude/skills/.
$ gh skill install guia-matthieu/clawfu-skills ab-test-stats --agent claude-codeProject scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).
$ git clone --depth 1 https://github.com/guia-matthieu/clawfu-skills.git skills-src && mkdir -p .claude/skills && cp -r skills-src/skills/analytics/ab-test-stats .claude/skills/ab-test-stats && rm -rf skills-srcUse ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.
Claude Code skills documentation · loads skills from .claude/skills/
Install the "ab-test-stats" agent skill from https://github.com/guia-matthieu/clawfu-skills/tree/main/skills/analytics/ab-test-stats into .claude/skills/ab-test-stats/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "ab-test-stats", then confirm the skill loads.Claude Code copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$skill-installer install https://github.com/guia-matthieu/clawfu-skills/tree/main/skills/analytics/ab-test-statsType this inside Codex. $skill-installer <name> installs a curated skill from openai/skills. The installer writes to $CODEX_HOME/skills (default ~/.codex/skills). Restart Codex if the skill does not show up.
$ npx skills add guia-matthieu/clawfu-skills --skill ab-test-stats -a codexProject install goes to .agents/skills/; add -g for ~/.codex/skills/.
$ gh skill install guia-matthieu/clawfu-skills ab-test-stats --agent codexProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/guia-matthieu/clawfu-skills.git skills-src && mkdir -p .agents/skills && cp -r skills-src/skills/analytics/ab-test-stats .agents/skills/ab-test-stats && rm -rf skills-srcUse ~/.agents/skills/ instead of .agents/skills for a personal install.
Codex skills documentation · loads skills from .agents/skills/
Install the "ab-test-stats" agent skill from https://github.com/guia-matthieu/clawfu-skills/tree/main/skills/analytics/ab-test-stats into .agents/skills/ab-test-stats/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "ab-test-stats", then confirm the skill loads.Codex copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add guia-matthieu/clawfu-skills --skill ab-test-stats -a cursorProject install goes to .agents/skills/; add -g for ~/.cursor/skills/.
$ gh skill install guia-matthieu/clawfu-skills ab-test-stats --agent cursorProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/guia-matthieu/clawfu-skills.git skills-src && mkdir -p .cursor/skills && cp -r skills-src/skills/analytics/ab-test-stats .cursor/skills/ab-test-stats && rm -rf skills-srcUse ~/.cursor/skills/ instead of .cursor/skills for a personal install.
Cursor skills documentation · loads skills from .cursor/skills/, .agents/skills/, .claude/skills/, .codex/skills/
Install the "ab-test-stats" agent skill from https://github.com/guia-matthieu/clawfu-skills/tree/main/skills/analytics/ab-test-stats into .cursor/skills/ab-test-stats/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "ab-test-stats", then confirm the skill loads.Cursor copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gemini skills install https://github.com/guia-matthieu/clawfu-skills.git --path skills/analytics/ab-test-stats--scope user (default) or --scope workspace; --path is the subfolder of the repo that holds the skill; --consent skips the security confirmation prompt.
$ npx skills add guia-matthieu/clawfu-skills --skill ab-test-stats -a gemini-cliProject install goes to .agents/skills/; add -g for ~/.gemini/skills/.
$ gh skill install guia-matthieu/clawfu-skills ab-test-stats --agent gemini-cliProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/guia-matthieu/clawfu-skills.git skills-src && mkdir -p .gemini/skills && cp -r skills-src/skills/analytics/ab-test-stats .gemini/skills/ab-test-stats && rm -rf skills-srcUse ~/.gemini/skills/ instead of .gemini/skills for a personal install, then run /skills reload.
Gemini CLI skills documentation · loads skills from .gemini/skills/, .agents/skills/
Install the "ab-test-stats" agent skill from https://github.com/guia-matthieu/clawfu-skills/tree/main/skills/analytics/ab-test-stats into .gemini/skills/ab-test-stats/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "ab-test-stats", then confirm the skill loads.Gemini CLI copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gh skill install guia-matthieu/clawfu-skills ab-test-statsInstalls for Copilot at project scope by default; add --scope user for a personal install. Preview a skill first with gh skill preview. Needs GitHub CLI 2.90.0 or later (public preview).
$ npx skills add guia-matthieu/clawfu-skills --skill ab-test-stats -a github-copilotProject install goes to .agents/skills/; add -g for ~/.copilot/skills/.
$ git clone --depth 1 https://github.com/guia-matthieu/clawfu-skills.git skills-src && mkdir -p .github/skills && cp -r skills-src/skills/analytics/ab-test-stats .github/skills/ab-test-stats && rm -rf skills-srcUse ~/.copilot/skills/ instead of .github/skills for a personal install. Commit .github/skills so cloud agent and code review can use it.
GitHub Copilot skills documentation · loads skills from .github/skills/, .claude/skills/, .agents/skills/
Install the "ab-test-stats" agent skill from https://github.com/guia-matthieu/clawfu-skills/tree/main/skills/analytics/ab-test-stats into .github/skills/ab-test-stats/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "ab-test-stats", then confirm the skill loads.GitHub Copilot copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add guia-matthieu/clawfu-skills --skill ab-test-stats -a opencodeOpenCode documents no install command of its own. Project install goes to .agents/skills/; add -g for ~/.config/opencode/skills/.
$ gh skill install guia-matthieu/clawfu-skills ab-test-stats --agent opencodeProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/guia-matthieu/clawfu-skills.git skills-src && mkdir -p .opencode/skills && cp -r skills-src/skills/analytics/ab-test-stats .opencode/skills/ab-test-stats && rm -rf skills-srcUse ~/.config/opencode/skills/ instead of .opencode/skills for a personal install.
OpenCode skills documentation · loads skills from .opencode/skills/, .claude/skills/, .agents/skills/
Install the "ab-test-stats" agent skill from https://github.com/guia-matthieu/clawfu-skills/tree/main/skills/analytics/ab-test-stats into .opencode/skills/ab-test-stats/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "ab-test-stats", then confirm the skill loads.OpenCode copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
Facts
- Skill name
ab-test-stats- GitHub stars
- 150
- Token cost
- ~1k tokens
- SKILL.md length
- 226 words
- Files
- 3 (incl. scripts)
- Skills in repo
- 31
- Repo updated
- First seen
- Licence
- MIT
At a glance
Calculate A/B test statistical significance. An agent skill from guia-matthieu/clawfu-skills.
- : determining if test results are significant
- SKILL.md covers When to Use This Skill, What Claude Does vs What You…, Dependencies and Commands, plus 6 more sections
- Runs Python scripts from its folder; calls python and pip
- Calculating required sample size
What it does
Ab Test Stats is an agent skill from guia-matthieu/clawfu-skills. Calculate A/B test statistical significance. Use when: determining if test results are significant; calculating required sample size; estimating test duration; analyzing conversion experiments; making data-driven decisions
Its SKILL.md is about 1k tokens, which your agent loads only when the skill is triggered. The skill folder holds 3 other files, including scripts (for example `scripts/main.py`).
It sits in Marketing & SEO, covering A/B testing and Experimental design. The repository describes itself as: 172 expert marketing skills for AI agents — ClawFu MCP Server. The licence is MIT.
When your agent uses it
- : determining if test results are significant
- Calculating required sample size
- Estimating test duration
- Analyzing conversion experiments
Example prompts
- “/ab-test-stats”
Requirements
- Python 3
What it can do on your machine
Read from SKILL.md and the folder at commit 4108f5c. It shows what the files ask for, not the result of running them.
Tool permissions
Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.
From allowed-tools in the SKILL.md frontmatter.
Runs code
Ships 2 files in scripts/ (Python), which the agent can run.
Shell commands in SKILL.md call:
pythonpip
From the folder's file list and the shell code blocks in SKILL.md.
Network
No URLs in SKILL.md. Its commands use pip, which can reach the network depending on how they are called.
From URLs in SKILL.md, links to its own repository left out.
Credentials
Names no API keys, tokens, secrets or passwords.
From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.
Context cost
Ab Test Stats loads about 1k tokens when it runs. Until then it costs about 59 tokens; SKILL.md has 226 words of instructions outside code blocks.
Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.
Safety
Auto-check passed
The automated check found no risky patterns in SKILL.md.
Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); the scripts in this folder are not scanned.
SKILL.md
The full file from guia-matthieu/clawfu-skills at commit 4108f5c, republished under its MIT licence (© guia-matthieu). 226 words, ~1,013 tokens.
Download SKILL.mdSave it as
.claude/skills/ab-test-stats/SKILL.md (or your agent's skills folder). This skill also uses 2 other files; get the full folder from GitHub.- name
- ab-test-stats
- description
- Calculate A/B test statistical significance. Use when: determining if test results are significant; calculating required sample size; estimating test duration; analyzing conversion experiments; making data-driven decisions
- license
- MIT
- metadata.author
- ClawFu
- metadata.version
- 1.0.0
- metadata.mcp-server
- @clawfu/mcp-skills
A/B Test Statistics Calculator
Calculate statistical significance for A/B tests - know when your results are real, not random chance.
When to Use This Skill
- Test analysis - Determine if results are statistically significant
- Sample planning - Calculate required sample size before testing
- Duration estimation - Know how long to run experiments
- Power analysis - Ensure tests can detect meaningful differences
What Claude Does vs What You Decide
| Claude Does | You Decide |
|---|---|
| Structures analysis frameworks | Metric definitions |
| Identifies patterns in data | Business interpretation |
| Creates visualization templates | Dashboard design |
| Suggests optimization areas | Action priorities |
| Calculates statistical measures | Decision thresholds |
Dependencies
pip install scipy numpy clickCommands
Check Significance
python scripts/main.py significance --control 1000,50 --variant 1000,65
python scripts/main.py significance --control 5000,250 --variant 5000,300 --confidence 0.99Calculate Sample Size
python scripts/main.py sample-size --baseline 0.05 --mde 0.02
python scripts/main.py sample-size --baseline 0.10 --mde 0.01 --power 0.90Estimate Duration
python scripts/main.py duration --traffic 1000 --baseline 0.05 --mde 0.02Examples
Example 1: Analyze Test Results
# Control: 1000 visitors, 50 conversions (5%)
# Variant: 1000 visitors, 65 conversions (6.5%)
python scripts/main.py significance --control 1000,50 --variant 1000,65
# Output:
# A/B Test Results
# ─────────────────────────
# Control: 5.00% (50/1000)
# Variant: 6.50% (65/1000)
# Lift: +30.0%
#
# Statistical Analysis
# ─────────────────────────
# p-value: 0.089
# Confidence: 91.1%
# Result: NOT SIGNIFICANT (need 95%)
#
# Recommendation: Continue test for more dataExample 2: Plan Sample Size
# Baseline 5% conversion, want to detect 20% relative lift (1% absolute)
python scripts/main.py sample-size --baseline 0.05 --mde 0.01
# Output:
# Sample Size Calculator
# ──────────────────────────────
# Baseline conversion: 5.0%
# Minimum detectable effect: 1.0% (20% relative)
# Target conversion: 6.0%
#
# Required per variant: 3,842 visitors
# Total required: 7,684 visitors
#
# At 1000 daily visitors: ~8 daysKey Concepts
| Term | Definition |
|---|---|
| p-value | Probability result is due to chance |
| Confidence | 1 - p-value (usually want 95%+) |
| Power | Probability of detecting real effect (usually 80%) |
| MDE | Minimum Detectable Effect - smallest lift worth detecting |
| Lift | Relative improvement (variant - control) / control |
When Results Are Significant
| p-value | Confidence | Verdict |
|---|---|---|
| < 0.01 | > 99% | Highly Significant ✓ |
| < 0.05 | > 95% | Significant ✓ |
| < 0.10 | > 90% | Marginally Significant |
| ≥ 0.10 | < 90% | Not Significant ✗ |
Skill Boundaries
What This Skill Does Well
- Structuring data analysis
- Identifying patterns and trends
- Creating visualization frameworks
- Calculating statistical measures
What This Skill Cannot Do
- Access your actual data
- Replace statistical expertise
- Make business decisions
- Guarantee prediction accuracy
Related Skills
- cohort-analysis - Analyze user cohorts
- funnel-analyzer - Analyze conversion funnels
Skill Metadata
- Mode: centaur
category: analytics
subcategory: statistics
dependencies: [scipy, numpy]
difficulty: intermediate
time_saved: 3+ hours/week© guia-matthieu, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file
Files
SKILL.md and 2 other files (scripts) in skills/analytics/ab-test-stats of guia-matthieu/clawfu-skills.
- SKILL.md
- scripts/main.py
- scripts/requirements.txt
Open the folder on GitHubat commit 4108f5c
Compare with similar skills
Ab Test Stats next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.
| Skill | Stars | Used in | Tokens | Auto-check | Licence | Repo updated |
|---|---|---|---|---|---|---|
| Ab Test Stats this skillguia-matthieu/clawfu-skills | 150 | — | ~1k | Automated safety check: Pass | MIT | |
| Ad Test Designeraaron-he-zhu/aaron-marketing-skills | 2.9k | 2 repos | ~2.8k | Automated safety check: Pass | Apache-2.0 | |
| Ab Test Analyzeririnabuht12-oss/marketing-skills | 4.1k | — | ~1.4k | Automated safety check: Pass | None | |
| Define Hypothesisproduct-on-purpose/pm-skills | 716 | — | ~966 | Automated safety check: Pass | Apache-2.0 | |
| A B Test DesignOwl-Listener/designer-skills | 2.9k | 1 repos | ~472 | Automated safety check: Pass | MIT | |
| Ab Test Planindranilbanerjee/digital-marketing-pro | 862 | 1 repos | ~1.9k | Automated safety check: Pass | MIT |
Similar skills
Ab Test Analyzer
irinabuht12-oss/marketing-skills
Statistical significance calculator for A/B test results with sample size requirements, segment breakdowns, and hypothesis generation.
Define Hypothesis
product-on-purpose/pm-skills
Defines a testable hypothesis with clear success metrics and a validation approach.
A B Test Design
Owl-Listener/designer-skills
Design an A/B experiment — hypothesis, variants, primary metric, and sample size.
Ab Test Plan
indranilbanerjee/digital-marketing-pro
Plan an A/B test by script: sample size per variant, days to run, stopping rules.
Ab Testing
ericrisco/rsc-harness
A skill your agent uses when designing or analyzing a controlled experiment — falsifiable hypothesis, sample size from an MDE, reading significance/CI/power, CUPED, or rescuing tests that won't go…
More from guia-matthieu/clawfu-skills
All 31 skills in this repoCohort Analysis
guia-matthieu/clawfu-skills
Analyze user retention by cohort. An agent skill from guia-matthieu/clawfu-skills.
Competitive Analysis
guia-matthieu/clawfu-skills
Analyze your competitive landscape using Porter's Five Forces and modern frameworks—understand industry dynamics, identify strategic opportunities, and position your business for sustainable…
Competitor Monitor
guia-matthieu/clawfu-skills
Monitor competitor websites for changes. An agent skill from guia-matthieu/clawfu-skills.
Content Repurposer
guia-matthieu/clawfu-skills
Transform long-form content into multiple short-form pieces.
Deliverability Checker
guia-matthieu/clawfu-skills
Check email deliverability and DNS configuration. An agent skill from guia-matthieu/clawfu-skills.
Image Batch
guia-matthieu/clawfu-skills
Batch process images for marketing. An agent skill from guia-matthieu/clawfu-skills.
Related
Categories
Questions about Ab Test Stats
What does Ab Test Stats do?
Calculate A/B test statistical significance. An agent skill from guia-matthieu/clawfu-skills. Ab Test Stats is an agent skill from guia-matthieu/clawfu-skills. Calculate A/B test statistical significance.
When should I use Ab Test Stats?
Ab Test Stats fits situations like: : determining if test results are significant; calculating required sample size; estimating test duration; analyzing conversion experiments.
How do I install Ab Test Stats in Claude Code?
Run `npx skills add guia-matthieu/clawfu-skills --skill ab-test-stats -a claude-code`. Or copy the skill folder (skills/analytics/ab-test-stats in guia-matthieu/clawfu-skills) into .claude/skills/ab-test-stats in your project. Claude Code loads it when a task matches its description.
How do I install Ab Test Stats in Codex?
Run `npx skills add guia-matthieu/clawfu-skills --skill ab-test-stats -a codex`. Or copy the skill folder (skills/analytics/ab-test-stats in guia-matthieu/clawfu-skills) into .agents/skills/ab-test-stats in your project. Codex loads it when a task matches its description.
Can I use Ab Test Stats in Cursor, Gemini CLI or GitHub Copilot?
Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add guia-matthieu/clawfu-skills --skill ab-test-stats -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/ab-test-stats, .gemini/skills/ab-test-stats, .github/skills/ab-test-stats and .opencode/skills/ab-test-stats in your project.
What does Ab Test Stats need to run?
Going by SKILL.md and its folder, Ab Test Stats needs Python for the scripts in its folder and the command-line tools its instructions call (python and pip). Our summary lists: Python 3.
Does Ab Test Stats access the network?
SKILL.md contains no URLs. Its commands use pip, which can reach the network depending on how they are called. This is read from the text; nothing was executed.
Is Ab Test Stats safe to install?
Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. The check reads SKILL.md only: the scripts in the folder are not scanned, so read them before running anything.
What licence does Ab Test Stats use?
Ab Test Stats is published under the MIT licence (declared in SKILL.md). It allows redistribution, so the full SKILL.md is shown on this page.
How many tokens does Ab Test Stats use?
About 1k tokens (SKILL.md is roughly 4.1k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.
What are the alternatives to Ab Test Stats?
Skills that share tags, products or a category with Ab Test Stats: Ad Test Designer (aaron-he-zhu/aaron-marketing-skills, 2.9k stars), Ab Test Analyzer (irinabuht12-oss/marketing-skills, 4.1k stars), Define Hypothesis (product-on-purpose/pm-skills, 716 stars) and A B Test Design (Owl-Listener/designer-skills, 2.9k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.
Who maintains Ab Test Stats?
guia-matthieu (a GitHub user) maintains it in guia-matthieu/clawfu-skills, which has 150 GitHub stars. The repository holds 31 skills in this directory. The repository was last updated on October 1, 2026.
Source: guia-matthieu/clawfu-skills on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.