New: 98,000+ skills indexed from GitHub repos with 100+ stars · every skill safety-checkedBrowse top skillsTop skills
Esc to closeBrowse all skills
Submit a skillBrowse skills
Submit a skillBrowse skills

Agent skill

A B Test Design

by Owl-Listener in Owl-Listener/designer-skills

Design an A/B experiment — hypothesis, variants, primary metric, and sample size.

MITAuto-check passedMarketing & SEO

Install A B Test Design

skills CLI
$ npx skills add Owl-Listener/designer-skills --skill a-b-test-design -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install Owl-Listener/designer-skills a-b-test-design --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/Owl-Listener/designer-skills.git skills-src && mkdir -p .claude/skills && cp -r skills-src/prototyping-testing/skills/a-b-test-design .claude/skills/a-b-test-design && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
a-b-test-design
GitHub stars
2.9k
Used in
1 other repo
Token cost
~472 tokens
SKILL.md length
222 words
Files
1
Skills in repo
107
Repo updated
First seen
Licence
MIT

At a glance

Design an A/B experiment — hypothesis, variants, primary metric, and sample size.

  • Works in 6 steps: Hypothesis → Variants → Primary Metric → …
  • A change can be measured quantitatively at scale
  • SKILL.md covers What You Do, Test Structure, Common Pitfalls and When Not to A/B Test, plus 1 more section
  • Instructions only: no scripts, shell commands, URLs or credentials in SKILL.md

What it does

A B Test Design is an agent skill from Owl-Listener/designer-skills. Design an A/B experiment — hypothesis, variants, primary metric, and sample size. Use when a change can be measured quantitatively at scale. For observing behaviour qualitatively, use test-scenario.

Its SKILL.md is about 470 tokens, which your agent loads only when the skill is triggered. It is a single SKILL.md file with no bundled scripts.

It sits in Marketing & SEO, covering A/B testing and Experimental design. The repository describes itself as: Designer Skills Collection: agentic skills, commands, and plugins for design — from research to systems, UI, interaction, and delivery. The licence is MIT.

When your agent uses it

  • A change can be measured quantitatively at scale
  • Tasks that involve A/B testing
  • Tasks that involve Experimental design

Example prompts

  • “/a-b-test-design”

Workflow steps

6 steps, taken from the step headings in SKILL.md.

  1. Hypothesis
  2. Variants
  3. Primary Metric
  4. Secondary Metrics
  5. Sample Size
  6. Duration

What it can do on your machine

Read from SKILL.md and the folder at commit 9a6930c. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    No scripts in the folder and no shell commands in SKILL.md.

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    No URLs in SKILL.md.

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

A B Test Design loads about 472 tokens when it runs. Until then it costs about 54 tokens; SKILL.md has 222 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~54
When it runs · the whole SKILL.md, loaded when a task matches
~472

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from Owl-Listener/designer-skills at commit 9a6930c, republished under its MIT licence (© Owl-Listener). 222 words, ~472 tokens.

Download SKILL.mdSave it as .claude/skills/a-b-test-design/SKILL.md (or your agent's skills folder).
name
a-b-test-design
description
Design an A/B experiment — hypothesis, variants, primary metric, and sample size. Use when a change can be measured quantitatively at scale. For observing behaviour qualitatively, use `test-scenario`.

A/B Test Design

You are an expert in designing rigorous A/B experiments that produce actionable results.

What You Do

You design A/B tests with clear hypotheses, controlled variants, appropriate metrics, and statistical rigor.

Test Structure

1. Hypothesis

Structured as: 'If we [change], then [outcome] will [improve/decrease] because [rationale].'

2. Variants
  • Control (A): current design
  • Treatment (B): proposed change
  • Keep changes isolated — test one variable at a time
3. Primary Metric

The single most important measure of success. Must be measurable, relevant, and sensitive to the change.

4. Secondary Metrics

Supporting measures and guardrail metrics to detect unintended consequences.

5. Sample Size

Based on: minimum detectable effect, baseline conversion rate, statistical significance level (typically 95%), and power (typically 80%).

6. Duration

Run until sample size is reached. Account for weekly cycles (run in full weeks). Minimum 1-2 weeks typically.

Common Pitfalls

  • Peeking at results before completion
  • Too many variants at once
  • Metric not sensitive enough to detect change
  • Sample size too small
  • Not accounting for novelty effects
  • Ignoring segmentation effects

When Not to A/B Test

  • Very low traffic (insufficient sample)
  • Ethical concerns with withholding improvement
  • Foundational changes that affect everything
  • When qualitative insight is more valuable

Best Practices

  • One hypothesis per test
  • Document everything before starting
  • Don't stop early on positive results
  • Analyze segments after overall results
  • Share learnings broadly regardless of outcome

© Owl-Listener, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

Just SKILL.md in prototyping-testing/skills/a-b-test-design of Owl-Listener/designer-skills.

Open the folder on GitHubat commit 9a6930c

Used in 1 other repository

We found 2 copies of this SKILL.md (exact, near-identical or edited) in other folders, from 1 other GitHub owner. This page covers the copy in Owl-Listener/designer-skills, which our catalogue first saw on October 7, 2026.

Compare with similar skills

A B Test Design next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

A B Test Design compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
A B Test Design this skillOwl-Listener/designer-skills2.9k1 repos~472Automated safety check: PassMIT
Ab Test Analyzeririnabuht12-oss/marketing-skills4.1k—~1.4kAutomated safety check: PassNone
Define Hypothesisproduct-on-purpose/pm-skills716—~966Automated safety check: PassApache-2.0
Send Experiment Designeraaron-he-zhu/aaron-marketing-skills2.9k2 repos~4.1kAutomated safety check: PassApache-2.0
Content Experimentation Best Practicessanity-io/agent-toolkit1881 repos~469Automated safety check: PassMIT
Ads TestAgriciDaniel/claude-ads9.9k—~312Automated safety check: PassMIT

Similar skills

  • Ab Test Analyzer

    irinabuht12-oss/marketing-skills

    Statistical significance calculator for A/B test results with sample size requirements, segment breakdowns, and hypothesis generation.

    4.1k GitHub stars~1.4k tokensUpdated 16 days ago
    Marketing & SEOAuto-check passed

  • Define Hypothesis

    product-on-purpose/pm-skills

    Defines a testable hypothesis with clear success metrics and a validation approach.

    716 GitHub stars~966 tokensUpdated 2 days ago
    Marketing & SEOAuto-check passed

  • Send Experiment Designer

    aaron-he-zhu/aaron-marketing-skills

    A skill your agent uses when the user asks to "design an email A/B test", "set up a multivariate subject/CTA test", "run a send-time test", "build a hold-out group", or "is this email result…

    2.9k GitHub starsUsed in 2 repos~4.1k tokens
    Marketing & SEOAuto-check passed

  • Content Experimentation Best Practices

    sanity-io/agent-toolkit

    Official

    Content experimentation and A/B testing guidance covering experiment design, hypotheses, metrics, sample size, statistical foundations, CMS-managed variants, and common analysis pitfalls.

    188 GitHub starsUsed in 1 repo~469 tokens
    Marketing & SEOAuto-check passed

  • Ads Test

    AgriciDaniel/claude-ads

    Design and evaluate paid-ad experiments with hypotheses, randomization units, sample-size and duration assumptions, guardrails, platform experiment tools, analysis, and decision rules.

    9.9k GitHub stars~312 tokensUpdated 2 days ago
    Marketing & SEOAuto-check passed

  • Ab Test Setup

    sickn33/agentic-awesome-skills

    A skill your agent uses when designing an A/B or split test: define the hypothesis, control and variants, estimate sample size, verify tracking, and predeclare metrics and stopping rules.

    47k GitHub starsUsed in 1 repo~2.2k tokens
    Marketing & SEOAuto-check passed

More from Owl-Listener/designer-skills

All 107 skills in this repo

  • Accessibility Test Plan

    Owl-Listener/designer-skills

    Plan accessibility testing — assistive technologies, participant criteria, WCAG coverage, and session protocol.

    2.9k GitHub starsUsed in 1 repo~478 tokens
    Auto-check passed

  • Aesthetic Usability

    Owl-Listener/designer-skills

    Apply the Aesthetic-Usability Effect — polished, consistent interfaces are perceived as more usable and forgive minor friction.

    2.9k GitHub starsUsed in 1 repo~678 tokens
    Auto-check passed

  • Affinity Diagram

    Owl-Listener/designer-skills

    Cluster many qualitative data points into themes and insight statements.

    2.9k GitHub starsUsed in 1 repo~520 tokens
    Auto-check passed

  • Animation Principles

    Owl-Listener/designer-skills

    Apply animation principles — easing, staging, follow-through — to one specific UI motion.

    2.9k GitHub starsUsed in 1 repo~492 tokens
    Auto-check passed

  • Business Design

    Owl-Listener/designer-skills

    Read financials, map competitive landscapes, and argue design decisions in the language of value.

    2.9k GitHub starsUsed in 1 repo~1.1k tokens
    Auto-check passed

  • Case Study

    Owl-Listener/designer-skills

    Craft a portfolio case study with narrative arc, process evidence, and outcomes.

    2.9k GitHub starsUsed in 1 repo~523 tokens
    Auto-check passed

Categories

Questions about A B Test Design

What does A B Test Design do?

Design an A/B experiment — hypothesis, variants, primary metric, and sample size. A B Test Design is an agent skill from Owl-Listener/designer-skills. Design an A/B experiment — hypothesis, variants, primary metric, and sample size.

When should I use A B Test Design?

A B Test Design fits situations like: A change can be measured quantitatively at scale; tasks that involve A/B testing; tasks that involve Experimental design.

How do I install A B Test Design in Claude Code?

Run `npx skills add Owl-Listener/designer-skills --skill a-b-test-design -a claude-code`. Or copy the skill folder (prototyping-testing/skills/a-b-test-design in Owl-Listener/designer-skills) into .claude/skills/a-b-test-design in your project. Claude Code loads it when a task matches its description.

How do I install A B Test Design in Codex?

Run `npx skills add Owl-Listener/designer-skills --skill a-b-test-design -a codex`. Or copy the skill folder (prototyping-testing/skills/a-b-test-design in Owl-Listener/designer-skills) into .agents/skills/a-b-test-design in your project. Codex loads it when a task matches its description.

Can I use A B Test Design in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add Owl-Listener/designer-skills --skill a-b-test-design -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/a-b-test-design, .gemini/skills/a-b-test-design, .github/skills/a-b-test-design and .opencode/skills/a-b-test-design in your project.

What does A B Test Design need to run?

SKILL.md names no scripts, command-line tools or credentials: A B Test Design is instructions for the agent only.

Does A B Test Design access the network?

SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.

Is A B Test Design safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.

What licence does A B Test Design use?

A B Test Design is published under the MIT licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does A B Test Design use?

About 472 tokens (SKILL.md is roughly 1.9k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.

What are the alternatives to A B Test Design?

Skills that share tags, products or a category with A B Test Design: Ab Test Analyzer (irinabuht12-oss/marketing-skills, 4.1k stars), Define Hypothesis (product-on-purpose/pm-skills, 716 stars), Send Experiment Designer (aaron-he-zhu/aaron-marketing-skills, 2.9k stars) and Content Experimentation Best Practices (sanity-io/agent-toolkit, 188 stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains A B Test Design?

Owl-Listener (a GitHub user) maintains it in Owl-Listener/designer-skills, which has 2,871 GitHub stars. The repository holds 107 skills in this directory. The repository was last updated on September 5, 2026.

Source: Owl-Listener/designer-skills on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.