Agent skill

Analyze Experiments

by amplitude in amplitude/builder-skills

Designs A/B tests with proper metrics and variants, analyzes running or completed experiments, and interprets results with statistical rigor.

No licenceAuto-check passedMarketing & SEO

Install Analyze Experiments

skills CLI
$ npx skills add amplitude/builder-skills --skill analyze-experiments -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install amplitude/builder-skills analyze-experiments --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/amplitude/builder-skills.git skills-src && mkdir -p .claude/skills && cp -r skills-src/analytics-skills/skills/analyze-experiment .claude/skills/analyze-experiments && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
analyze-experiments
GitHub stars
159
Token cost
~5.4k tokens
SKILL.md length
2,595 words
Files
1
Skills in repo
61
Repo updated
First seen
Licence
None found

At a glance

Designs A/B tests with proper metrics and variants, analyzes running or completed experiments, and interprets results with statistical rigor.

  • Works in 9 steps: Identify Experiment → Retrieve and Validate Setup → Check Data Quality (with explicit… → …
  • Setting up experiments
  • SKILL.md covers When to Use, Analysis Philosophy, Instructions and Experiment Analysis:…, plus 3 more sections
  • Instructions only: no scripts, shell commands, URLs or credentials in SKILL.md

What it does

Analyze Experiments is an agent skill from amplitude/builder-skills. Designs A/B tests with proper metrics and variants, analyzes running or completed experiments, and interprets results with statistical rigor. Use when setting up experiments, checking experiment status, analyzing results, or making ship decisions.

Its SKILL.md is about 5.4k tokens, which your agent loads only when the skill is triggered. It is a single SKILL.md file with no bundled scripts.

It sits in Marketing & SEO, covering A/B testing. The repository describes itself as: Open-source AI skills for product managers — battle-tested by PMs at Amplitude.

When your agent uses it

  • Setting up experiments
  • Checking experiment status
  • Analyzing results
  • Making ship decisions

Example prompts

  • “Use the analyze-experiments skill to design A/B tests with proper metrics and variants, analyzes running or completed experiments, and interprets…”
  • “/analyze-experiments”

Workflow steps

9 steps, taken from the step headings in SKILL.md.

  1. Identify Experiment
  2. Retrieve and Validate Setup
  3. Check Data Quality (with explicit thresholds)
  4. Analyze Primary Metric
  5. Analyze Secondary Metrics & Guardrails
  6. Comprehensive Segment Analysis
  7. Assess Duration & Runtime Sufficiency
  8. Understand Why (Qualitative Context)
  9. Synthesize Findings and Make Recommendation

What it can do on your machine

Read from SKILL.md and the folder at commit 51838c2. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    No scripts in the folder and no shell commands in SKILL.md.

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    No URLs in SKILL.md.

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Analyze Experiments loads about 5.4k tokens when it runs. Until then it costs about 67 tokens; SKILL.md has 2,595 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~67
When it runs · the whole SKILL.md, loaded when a task matches
~5.4k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

Without a licence we can't republish the file, so here is its outline and opening line. It has 2,595 words (~5,445 tokens).

“Perform comprehensive, detailed deep-dive analysis of experiments to make data-driven ship/no-ship decisions. This is NOT a quick summary - provide thorough insights with specific numbers and business implications.”

— opening of SKILL.md by amplitude
name
analyze-experiments
suggest_when
User asks about a specific experiment, shares an experiment URL, asks "did this test win", "should we ship this", or wants statistical analysis of A/B test…

Read the full SKILL.md on GitHub

Files

Just SKILL.md in analytics-skills/skills/analyze-experiment of amplitude/builder-skills.

Open the folder on GitHubat commit 51838c2

Compare with similar skills

Analyze Experiments next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Analyze Experiments compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Analyze Experiments this skillamplitude/builder-skills159—~5.4kAutomated safety check: PassNone
AnalyticsNexus-JPF/note-companion8707 repos~2.2kAutomated safety check: PassMIT
App Analyticsappeeky/aso-skills2.2k—~1.6kAutomated safety check: PassMIT
Ab Test Analysisnimrodfisher/data-analytics-skills465—~708Automated safety check: PassMIT
Ab Testingericrisco/rsc-harness167—~2.4kAutomated safety check: PassMIT
Bio Hi C Analysis Hic DifferentialFreedomIntelligence/OpenClaw-Medical-Skills3.1k1 repos~2.6kAutomated safety check: PassNone

Similar skills

  • Analytics

    Nexus-JPF/note-companion

    When the user wants to set up, improve, or audit analytics tracking and measurement.

    870 GitHub starsUsed in 7 repos~2.2k tokens
    Marketing & SEOAuto-check passed
  • App Analytics

    appeeky/aso-skills

    When the user wants to set up, interpret, or improve their app analytics and tracking.

    2.2k GitHub stars~1.6k tokensUpdated yesterday
    Marketing & SEOAuto-check passed
  • Ab Test Analysis

    nimrodfisher/data-analytics-skills

    Rigorous A/B test statistical analysis. An agent skill from nimrodfisher/data-analytics-skills.

    465 GitHub stars~708 tokensUpdated 12 days ago
    Marketing & SEOAuto-check passed
  • Ab Testing

    ericrisco/rsc-harness

    A skill your agent uses when designing or analyzing a controlled experiment — falsifiable hypothesis, sample size from an MDE, reading significance/CI/power, CUPED, or rescuing tests that won't go…

    167 GitHub stars~2.4k tokensUpdated today
    Marketing & SEOAuto-check passed
  • Bio Hi C Analysis Hic Differential

    FreedomIntelligence/OpenClaw-Medical-Skills

    Compare Hi-C contact matrices between conditions to identify differential chromatin interactions.

    3.1k GitHub starsUsed in 1 repo~2.6k tokens
    Marketing & SEOAuto-check passed
  • Measure Experiment Results

    product-on-purpose/pm-skills

    Documents the results of a completed experiment or A/B test with statistical analysis, learnings, and recommendations.

    715 GitHub stars~989 tokensUpdated 3 days ago
    Marketing & SEOAuto-check passed

More from amplitude/builder-skills

All 61 skills in this repo
  • 7 Powers Analysis

    amplitude/builder-skills

    Analyzes a business, product or feature with Hamilton Helmer's 7 Powers framework to judge competitive advantage and how long it will last.

    159 GitHub stars~1.1k tokensUpdated 2 mo ago
    Auto-check passed
  • Amplitude Custom Agent Builder

    amplitude/builder-skills

    Helps an Amplitude user design a custom agent by proposing use cases that fit their role and data, shaping a spec, and returning a Global Agent link that builds it.

    159 GitHub stars~2.5k tokensUpdated 2 mo ago
    Auto-check passed
  • Good Strategy Bad Strategy Kernel

    amplitude/builder-skills

    Evaluates or drafts a strategy using Rumelt's kernel of diagnosis, guiding policy and coherent actions, to expose goal lists posing as strategy.

    159 GitHub stars~2.2k tokensUpdated 2 mo ago
    Auto-check passed
  • High Output Management Framework

    amplitude/builder-skills

    Applies Andy Grove's High Output Management ideas to diagnose where a team loses output and to design meetings, processes and management decisions.

    159 GitHub stars~1.9k tokensUpdated 2 mo ago
    Auto-check passed
  • Playing to Win Strategy

    amplitude/builder-skills

    Walks a team through Lafley and Martin's five-choice cascade to pin down where to compete and how to win there.

    159 GitHub stars~1.7k tokensUpdated 2 mo ago
    Auto-check passed
  • Amazon Working Backwards

    amplitude/builder-skills

    Write a PR/FAQ for a product idea using Amazon's Working Backwards process.

    159 GitHub stars~1.4k tokensUpdated 2 mo ago
    Auto-check passed

Questions about Analyze Experiments

What does Analyze Experiments do?

Designs A/B tests with proper metrics and variants, analyzes running or completed experiments, and interprets results with statistical rigor. Analyze Experiments is an agent skill from amplitude/builder-skills. Designs A/B tests with proper metrics and variants, analyzes running or completed experiments, and interprets results with statistical rigor.

When should I use Analyze Experiments?

Analyze Experiments fits situations like: setting up experiments; checking experiment status; analyzing results; making ship decisions.

How do I install Analyze Experiments in Claude Code?

Run `npx skills add amplitude/builder-skills --skill analyze-experiments -a claude-code`. Or copy the skill folder (analytics-skills/skills/analyze-experiment in amplitude/builder-skills) into .claude/skills/analyze-experiments in your project. Claude Code loads it when a task matches its description.

How do I install Analyze Experiments in Codex?

Run `npx skills add amplitude/builder-skills --skill analyze-experiments -a codex`. Or copy the skill folder (analytics-skills/skills/analyze-experiment in amplitude/builder-skills) into .agents/skills/analyze-experiments in your project. Codex loads it when a task matches its description.

Can I use Analyze Experiments in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add amplitude/builder-skills --skill analyze-experiments -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/analyze-experiments, .gemini/skills/analyze-experiments, .github/skills/analyze-experiments and .opencode/skills/analyze-experiments in your project.

What does Analyze Experiments need to run?

SKILL.md names no scripts, command-line tools or credentials: Analyze Experiments is instructions for the agent only.

Does Analyze Experiments access the network?

SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.

Is Analyze Experiments safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.

What licence does Analyze Experiments use?

No licence was found for Analyze Experiments or its repository. Without one, default copyright applies: ask the author before reusing or redistributing it.

How many tokens does Analyze Experiments use?

About 5.4k tokens (SKILL.md is roughly 22k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.

What are the alternatives to Analyze Experiments?

Skills that share tags, products or a category with Analyze Experiments: Analytics (Nexus-JPF/note-companion, 870 stars), App Analytics (appeeky/aso-skills, 2.2k stars), Ab Test Analysis (nimrodfisher/data-analytics-skills, 465 stars) and Ab Testing (ericrisco/rsc-harness, 167 stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Analyze Experiments?

amplitude (a GitHub organization) maintains it in amplitude/builder-skills, which has 159 GitHub stars. The repository holds 61 skills in this directory. The repository was last updated on July 24, 2026.

Source: amplitude/builder-skills on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.