Ab Testing
ericrisco/rsc-harness
A skill your agent uses when designing or analyzing a controlled experiment — falsifiable hypothesis, sample size from an MDE, reading significance/CI/power, CUPED, or rescuing tests that won't go…
Defines a testable hypothesis with clear success metrics and a validation approach.
Install Define Hypothesis
$ npx skills add product-on-purpose/pm-skills --skill define-hypothesis -a claude-codeProject install by default; add -g for ~/.claude/skills/.
$ gh skill install product-on-purpose/pm-skills define-hypothesis --agent claude-codeProject scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).
$ git clone --depth 1 https://github.com/product-on-purpose/pm-skills.git skills-src && mkdir -p .claude/skills && cp -r skills-src/skills/define-hypothesis .claude/skills/define-hypothesis && rm -rf skills-srcUse ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.
Claude Code skills documentation · loads skills from .claude/skills/
Install the "define-hypothesis" agent skill from https://github.com/product-on-purpose/pm-skills/tree/main/skills/define-hypothesis into .claude/skills/define-hypothesis/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "define-hypothesis", then confirm the skill loads.Claude Code copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$skill-installer install https://github.com/product-on-purpose/pm-skills/tree/main/skills/define-hypothesisType this inside Codex. $skill-installer <name> installs a curated skill from openai/skills. The installer writes to $CODEX_HOME/skills (default ~/.codex/skills). Restart Codex if the skill does not show up.
$ npx skills add product-on-purpose/pm-skills --skill define-hypothesis -a codexProject install goes to .agents/skills/; add -g for ~/.codex/skills/.
$ gh skill install product-on-purpose/pm-skills define-hypothesis --agent codexProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/product-on-purpose/pm-skills.git skills-src && mkdir -p .agents/skills && cp -r skills-src/skills/define-hypothesis .agents/skills/define-hypothesis && rm -rf skills-srcUse ~/.agents/skills/ instead of .agents/skills for a personal install.
Codex skills documentation · loads skills from .agents/skills/
Install the "define-hypothesis" agent skill from https://github.com/product-on-purpose/pm-skills/tree/main/skills/define-hypothesis into .agents/skills/define-hypothesis/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "define-hypothesis", then confirm the skill loads.Codex copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add product-on-purpose/pm-skills --skill define-hypothesis -a cursorProject install goes to .agents/skills/; add -g for ~/.cursor/skills/.
$ gh skill install product-on-purpose/pm-skills define-hypothesis --agent cursorProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/product-on-purpose/pm-skills.git skills-src && mkdir -p .cursor/skills && cp -r skills-src/skills/define-hypothesis .cursor/skills/define-hypothesis && rm -rf skills-srcUse ~/.cursor/skills/ instead of .cursor/skills for a personal install.
Cursor skills documentation · loads skills from .cursor/skills/, .agents/skills/, .claude/skills/, .codex/skills/
Install the "define-hypothesis" agent skill from https://github.com/product-on-purpose/pm-skills/tree/main/skills/define-hypothesis into .cursor/skills/define-hypothesis/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "define-hypothesis", then confirm the skill loads.Cursor copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gemini skills install https://github.com/product-on-purpose/pm-skills.git --path skills/define-hypothesis--scope user (default) or --scope workspace; --path is the subfolder of the repo that holds the skill; --consent skips the security confirmation prompt.
$ npx skills add product-on-purpose/pm-skills --skill define-hypothesis -a gemini-cliProject install goes to .agents/skills/; add -g for ~/.gemini/skills/.
$ gh skill install product-on-purpose/pm-skills define-hypothesis --agent gemini-cliProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/product-on-purpose/pm-skills.git skills-src && mkdir -p .gemini/skills && cp -r skills-src/skills/define-hypothesis .gemini/skills/define-hypothesis && rm -rf skills-srcUse ~/.gemini/skills/ instead of .gemini/skills for a personal install, then run /skills reload.
Gemini CLI skills documentation · loads skills from .gemini/skills/, .agents/skills/
Install the "define-hypothesis" agent skill from https://github.com/product-on-purpose/pm-skills/tree/main/skills/define-hypothesis into .gemini/skills/define-hypothesis/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "define-hypothesis", then confirm the skill loads.Gemini CLI copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gh skill install product-on-purpose/pm-skills define-hypothesisInstalls for Copilot at project scope by default; add --scope user for a personal install. Preview a skill first with gh skill preview. Needs GitHub CLI 2.90.0 or later (public preview).
$ npx skills add product-on-purpose/pm-skills --skill define-hypothesis -a github-copilotProject install goes to .agents/skills/; add -g for ~/.copilot/skills/.
$ git clone --depth 1 https://github.com/product-on-purpose/pm-skills.git skills-src && mkdir -p .github/skills && cp -r skills-src/skills/define-hypothesis .github/skills/define-hypothesis && rm -rf skills-srcUse ~/.copilot/skills/ instead of .github/skills for a personal install. Commit .github/skills so cloud agent and code review can use it.
GitHub Copilot skills documentation · loads skills from .github/skills/, .claude/skills/, .agents/skills/
Install the "define-hypothesis" agent skill from https://github.com/product-on-purpose/pm-skills/tree/main/skills/define-hypothesis into .github/skills/define-hypothesis/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "define-hypothesis", then confirm the skill loads.GitHub Copilot copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add product-on-purpose/pm-skills --skill define-hypothesis -a opencodeOpenCode documents no install command of its own. Project install goes to .agents/skills/; add -g for ~/.config/opencode/skills/.
$ gh skill install product-on-purpose/pm-skills define-hypothesis --agent opencodeProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/product-on-purpose/pm-skills.git skills-src && mkdir -p .opencode/skills && cp -r skills-src/skills/define-hypothesis .opencode/skills/define-hypothesis && rm -rf skills-srcUse ~/.config/opencode/skills/ instead of .opencode/skills for a personal install.
OpenCode skills documentation · loads skills from .opencode/skills/, .claude/skills/, .agents/skills/
Install the "define-hypothesis" agent skill from https://github.com/product-on-purpose/pm-skills/tree/main/skills/define-hypothesis into .opencode/skills/define-hypothesis/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "define-hypothesis", then confirm the skill loads.OpenCode copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
Facts
- Skill name
define-hypothesis- GitHub stars
- 715
- Token cost
- ~966 tokens
- SKILL.md length
- 438 words
- Files
- 5 (incl. references)
- Skills in repo
- 68
- Repo updated
- First seen
- Licence
- Apache-2.0
At a glance
Defines a testable hypothesis with clear success metrics and a validation approach.
- Works in 6 steps: State the Belief → Identify the Target User → Define the Expected Outcome → …
- Forming assumptions to test
- SKILL.md covers When to Use, When NOT to Use, Instructions and Output Format, plus 2 more sections
- Instructions only: no scripts, shell commands, URLs or credentials in SKILL.md
What it does
Define Hypothesis is an agent skill from product-on-purpose/pm-skills. Defines a testable hypothesis with clear success metrics and a validation approach. Use when forming assumptions to test or aligning a team on what success looks like, before any experiment is designed. To design the A/B test or experiment that will validate the hypothesis, use measure-experiment-design.
Its SKILL.md is about 970 tokens, which your agent loads only when the skill is triggered. The skill folder holds 6 other files, including reference files (for example `HISTORY.md`, `evals/trigger-fixtures.json` and `references/EXAMPLE.md`).
It sits in Marketing & SEO, covering A/B testing, Experimental design and Product metrics. The repository describes itself as: 68 plug-and-play, best-practice product management skills for AI agents: 30 Triple Diamond phase + 11 foundation + 12 utility + 15 tool (Foundation Sprint + Design Sprint). Plus… The licence is Apache-2.0.
When your agent uses it
- Forming assumptions to test
- Aligning a team on what success looks like
- Before any experiment is designed
Example prompts
- “Use the define-hypothesis skill to define a testable hypothesis with clear success metrics and a validation approach”
- “/define-hypothesis”
Workflow steps
6 steps, taken from the first numbered list in SKILL.md.
- State the Belief
- Identify the Target User
- Define the Expected Outcome
- Set Success Metrics
- Describe Validation Approach
- Document Risks and Assumptions
What it can do on your machine
Read from SKILL.md and the folder at commit 1cef1a9. It shows what the files ask for, not the result of running them.
Tool permissions
Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.
From allowed-tools in the SKILL.md frontmatter.
Runs code
No scripts in the folder and no shell commands in SKILL.md.
From the folder's file list and the shell code blocks in SKILL.md.
Network
No URLs in SKILL.md.
From URLs in SKILL.md, links to its own repository left out.
Credentials
Names no API keys, tokens, secrets or passwords.
From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.
Context cost
Define Hypothesis loads about 966 tokens when it runs, and up to ~3k if it reads all its reference files. Until then it costs about 81 tokens; SKILL.md has 438 words of instructions outside code blocks.
Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.
Safety
Auto-check passed
The automated check found no risky patterns in SKILL.md.
Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.
SKILL.md
The full file from product-on-purpose/pm-skills at commit 1cef1a9, republished under its Apache-2.0 licence (© product-on-purpose). 438 words, ~966 tokens.
Download SKILL.mdSave it as
.claude/skills/define-hypothesis/SKILL.md (or your agent's skills folder). This skill also uses 4 other files; get the full folder from GitHub.- name
- define-hypothesis
- description
- Defines a testable hypothesis with clear success metrics and a validation approach. Use when forming assumptions to test or aligning a team on what success looks like, before any experiment is designed. To design the A/B test or experiment that will validate the hypothesis, use measure-experiment-design.
- license
- Apache-2.0
- metadata.phase
- define
- metadata.version
- 2.1.0
- metadata.updated
- 2026-06-10
- metadata.category
- ideation
- metadata.frameworks
- triple-diamond, lean-startup, design-thinking
- metadata.author
- product-on-purpose
<!-- PM-Skills | https://github.com/product-on-purpose/pm-skills | Apache 2.0 -->Hypothesis
A hypothesis is a testable prediction about how a change will affect user behavior or business outcomes. It transforms assumptions into explicit statements that can be validated or invalidated through experimentation. Well-formed hypotheses prevent teams from building features based on untested beliefs and create shared understanding of what success looks like.
When to Use
- After problem framing, before committing to a solution
- When designing experiments or A/B tests
- When team members have differing assumptions about user behavior
- Before investing significant engineering resources in a feature
- When pivoting direction and need to validate the new approach
When NOT to Use
- You are ready to design the actual A/B test (variants, sample size, duration) -> use
measure-experiment-design; this skill frames what to test, not how - The problem itself is still unframed -> use
define-problem-statementfirst - You want to organize many assumptions and ideas into a discovery structure -> use
define-opportunity-tree - The team needs the full business-model picture, not one testable claim -> use
foundation-lean-canvas
Instructions
When asked to create a hypothesis, follow these steps:
State the Belief Articulate what you believe will happen. Use the structured format: "We believe that [action/change] for [target user] will [expected outcome]." Be specific about the intervention - vague hypotheses can't be tested.
Identify the Target User Define who this hypothesis applies to. A hypothesis about "users" is too broad. Specify the segment: new users in their first week, power users with 10+ sessions, churned users returning, etc.
Define the Expected Outcome What behavior change or result do you expect? Frame it in terms of user actions (complete onboarding, make a purchase, return within 7 days) rather than internal metrics when possible.
Set Success Metrics Choose a primary metric that directly measures the expected outcome. Include secondary metrics that provide context and guardrail metrics that ensure you're not causing harm elsewhere.
Describe Validation Approach How will you test this hypothesis? A/B test, user interviews, prototype testing, cohort analysis? Be specific about sample size, duration, and statistical requirements.
Document Risks and Assumptions What could invalidate this hypothesis beyond the test results? What are you assuming to be true that you haven't validated?
Show full SKILL.md (82 more words)Show less
Output Format
Use the template in references/TEMPLATE.md to structure the output. A complete hypothesis document fills every template section: Hypothesis Statement; Background & Rationale; Target User Segment; Success Metrics; Validation Approach; Risks & Assumptions; and Timeline.
Quality Checklist
Before finalizing, verify:
- Hypothesis is falsifiable (possible to prove wrong)
- Success metric has a specific numeric target
- Target user segment is clearly defined
- Validation approach is practical and time-bound
- Pass/fail criteria are unambiguous
- Hypothesis doesn't assume the solution works
Examples
See references/EXAMPLE.md for a completed example.
© product-on-purpose, Apache-2.0. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file
Files
SKILL.md and 4 other files (references) in skills/define-hypothesis of product-on-purpose/pm-skills.
- SKILL.md
- HISTORY.md
- evals/trigger-fixtures.json
- references/EXAMPLE.md
- references/TEMPLATE.md
Open the folder on GitHubat commit 1cef1a9
Compare with similar skills
Define Hypothesis next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.
| Skill | Stars | Used in | Tokens | Auto-check | Licence | Repo updated |
|---|---|---|---|---|---|---|
| Define Hypothesis this skillproduct-on-purpose/pm-skills | 715 | — | ~966 | Automated safety check: Pass | Apache-2.0 | |
| Ab Testingericrisco/rsc-harness | 174 | — | ~2.4k | Automated safety check: Pass | MIT | |
| Craft Experiment Designamplitude/builder-skills | 160 | — | ~522 | Automated safety check: Pass | None | |
| Ab Test Analyzeririnabuht12-oss/marketing-skills | 4k | — | ~1.4k | Automated safety check: Pass | None | |
| A B Test DesignOwl-Listener/designer-skills | 2.9k | 1 repos | ~472 | Automated safety check: Pass | MIT | |
| Send Experiment Designeraaron-he-zhu/aaron-marketing-skills | 2.9k | 2 repos | ~4.1k | Automated safety check: Pass | Apache-2.0 |
Similar skills
Craft Experiment Design
amplitude/builder-skills
Write a hypothesis, define success metrics, and plan a holdout strategy.
Ab Test Analyzer
irinabuht12-oss/marketing-skills
Statistical significance calculator for A/B test results with sample size requirements, segment breakdowns, and hypothesis generation.
A B Test Design
Owl-Listener/designer-skills
Design an A/B experiment — hypothesis, variants, primary metric, and sample size.
Send Experiment Designer
aaron-he-zhu/aaron-marketing-skills
A skill your agent uses when the user asks to "design an email A/B test", "set up a multivariate subject/CTA test", "run a send-time test", "build a hold-out group", or "is this email result…
- Official
Content Experimentation Best Practices
sanity-io/agent-toolkit
Content experimentation and A/B testing guidance covering experiment design, hypotheses, metrics, sample size, statistical foundations, CMS-managed variants, and common analysis pitfalls.
More from product-on-purpose/pm-skills
All 68 skills in this repoDefine Jtbd Canvas
product-on-purpose/pm-skills
Creates a Jobs to be Done canvas capturing the functional, emotional, and social dimensions of a customer job.
Define Opportunity Tree
product-on-purpose/pm-skills
Creates an opportunity solution tree connecting a desired outcome to customer opportunities and candidate solutions, preventing solution-first jumps in continuous discovery.
Define Problem Statement
product-on-purpose/pm-skills
Creates a clear problem framing document with user impact, business context, and success criteria.
Deliver Acceptance Criteria
product-on-purpose/pm-skills
Generates structured Given/When/Then acceptance criteria for a user story or feature slice, covering the happy path, key failure scenarios, and non-functional expectations in testable form.
Deliver Launch Checklist
product-on-purpose/pm-skills
Creates a cross-functional pre-launch checklist covering engineering, design, marketing, support, legal, and operations readiness, with owners, dates, and go/no-go criteria so nothing is missed…
Deliver Prd
product-on-purpose/pm-skills
Creates a comprehensive Product Requirements Document that aligns stakeholders on what to build, why, and how success will be measured.
Related
Questions about Define Hypothesis
What does Define Hypothesis do?
Defines a testable hypothesis with clear success metrics and a validation approach. Define Hypothesis is an agent skill from product-on-purpose/pm-skills. Defines a testable hypothesis with clear success metrics and a validation approach.
When should I use Define Hypothesis?
Define Hypothesis fits situations like: forming assumptions to test; aligning a team on what success looks like; before any experiment is designed.
How do I install Define Hypothesis in Claude Code?
Run `npx skills add product-on-purpose/pm-skills --skill define-hypothesis -a claude-code`. Or copy the skill folder (skills/define-hypothesis in product-on-purpose/pm-skills) into .claude/skills/define-hypothesis in your project. Claude Code loads it when a task matches its description.
How do I install Define Hypothesis in Codex?
Run `npx skills add product-on-purpose/pm-skills --skill define-hypothesis -a codex`. Or copy the skill folder (skills/define-hypothesis in product-on-purpose/pm-skills) into .agents/skills/define-hypothesis in your project. Codex loads it when a task matches its description.
Can I use Define Hypothesis in Cursor, Gemini CLI or GitHub Copilot?
Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add product-on-purpose/pm-skills --skill define-hypothesis -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/define-hypothesis, .gemini/skills/define-hypothesis, .github/skills/define-hypothesis and .opencode/skills/define-hypothesis in your project.
What does Define Hypothesis need to run?
SKILL.md names no scripts, command-line tools or credentials: Define Hypothesis is instructions for the agent only.
Does Define Hypothesis access the network?
SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.
Is Define Hypothesis safe to install?
Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.
What licence does Define Hypothesis use?
Define Hypothesis is published under the Apache-2.0 licence (declared in SKILL.md). It allows redistribution, so the full SKILL.md is shown on this page.
How many tokens does Define Hypothesis use?
About 966 tokens (SKILL.md is roughly 3.9k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full. Its references folder adds about 2.1k tokens, read only when the agent opens those files.
What are the alternatives to Define Hypothesis?
Skills that share tags, products or a category with Define Hypothesis: Ab Testing (ericrisco/rsc-harness, 174 stars), Craft Experiment Design (amplitude/builder-skills, 160 stars), Ab Test Analyzer (irinabuht12-oss/marketing-skills, 4k stars) and A B Test Design (Owl-Listener/designer-skills, 2.9k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.
Who maintains Define Hypothesis?
product-on-purpose (a GitHub organization) maintains it in product-on-purpose/pm-skills, which has 715 GitHub stars. The repository holds 68 skills in this directory. The repository was last updated on October 8, 2026.
Source: product-on-purpose/pm-skills on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.