New: 97,000+ skills indexed from GitHub repos with 100+ stars · every skill safety-checkedBrowse top skillsTop skills
Esc to closeBrowse all skills
Submit a skillBrowse skills
Submit a skillBrowse skills

Agent skill

Ab Test Analysis

by killvxk in killvxk/pm-skills-zh

分析 A/B 测试结果，涵盖统计显著性检验、样本量验证、置信区间计算及上线/延长/停止的决策建议。适用于评估实验结果、判断测试是否达到显著性、解读分流测试数据，或决定是否上线某个实验组。

MITAuto-check passedMarketing & SEO

Install Ab Test Analysis

skills CLI
$ npx skills add killvxk/pm-skills-zh --skill ab-test-analysis -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install killvxk/pm-skills-zh ab-test-analysis --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/killvxk/pm-skills-zh.git skills-src && mkdir -p .claude/skills && cp -r skills-src/pm-data-analytics/skills/ab-test-analysis .claude/skills/ab-test-analysis && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
ab-test-analysis
GitHub stars
167
Token cost
~480 tokens
SKILL.md length
109 words
Files
1
Skills in repo
11
Repo updated
First seen
Licence
MIT

At a glance

分析 A/B 测试结果，涵盖统计显著性检验、样本量验证、置信区间计算及上线/延长/停止的决策建议。适用于评估实验结果、判断测试是否达到显著性、解读分流测试数据，或决定是否上线某个实验组。

  • Works in 6 steps: 理解实验背景 → 验证测试设置 → 计算统计显著性 → …
  • Tasks that involve A/B testing
  • Instructions only: no scripts, shell commands, URLs or credentials in SKILL.md

What it does

Ab Test Analysis is an agent skill from killvxk/pm-skills-zh. 分析 A/B 测试结果，涵盖统计显著性检验、样本量验证、置信区间计算及上线/延长/停止的决策建议。适用于评估实验结果、判断测试是否达到显著性、解读分流测试数据，或决定是否上线某个实验组。

Its SKILL.md is about 480 tokens, which your agent loads only when the skill is triggered. It is a single SKILL.md file with no bundled scripts.

It sits in Marketing & SEO, covering A/B testing. It works with Python. The repository describes itself as: PM Skills Marketplace 简体中文版 - 65个产品经理技能和36个工作流，翻译自 https://github.com/phuryn/pm-skills/. The licence is MIT.

When your agent uses it

  • Tasks that involve A/B testing

Example prompts

  • “/ab-test-analysis”

Requirements

  • Python 3

Workflow steps

6 steps, taken from the first numbered list in SKILL.md.

  1. 理解实验背景
  2. 验证测试设置
  3. 计算统计显著性
  4. 检查护栏指标
  5. 解读结果
  6. 提供分析摘要

What it can do on your machine

Read from SKILL.md and the folder at commit 5179784. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    No scripts in the folder and no shell commands in SKILL.md.

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    Links to these hosts (documentation or services it may open):

    • productcompass.pm

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Ab Test Analysis loads about 480 tokens when it runs. Until then it costs about 28 tokens; SKILL.md has 109 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~28
When it runs · the whole SKILL.md, loaded when a task matches
~480

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from killvxk/pm-skills-zh at commit 5179784, republished under its MIT licence (© killvxk). 109 words, ~480 tokens.

Download SKILL.mdSave it as .claude/skills/ab-test-analysis/SKILL.md (or your agent's skills folder).
name
ab-test-analysis
description
分析 A/B 测试结果，涵盖统计显著性检验、样本量验证、置信区间计算及上线/延长/停止的决策建议。适用于评估实验结果、判断测试是否达到显著性、解读分流测试数据，或决定是否上线某个实验组。

A/B 测试分析

以严格的统计方法评估 A/B 测试结果，并将发现转化为清晰的产品决策。

背景

你正在分析 $ARGUMENTS 的 A/B 测试结果。

如果用户提供了数据文件（CSV、Excel 或数据分析平台导出文件），直接读取并分析。如有需要，生成 Python 脚本执行统计计算。

操作步骤

  1. 理解实验背景：

    • 假设是什么？
    • 改动了什么（实验组的变化）？
    • 主要指标是什么？是否有护栏指标？
    • 测试运行了多长时间？
    • 流量分配比例是多少？

  2. 验证测试设置：

    • 样本量：样本量是否足以检测预期效应量？
      • 使用公式：n = (Z²α/2 × 2 × p × (1-p)) / MDE²
      • 如果统计功效不足（< 80%），需标记
    • 时长：测试是否至少运行了 1—2 个完整业务周期？
    • 随机化：是否有样本比例失配（SRM）的迹象？
    • 新奇效应/首因效应：时间是否足够让初期行为变化消退？

  3. 计算统计显著性：

    • 对照组和实验组各自的转化率
    • 相对提升：(实验组 - 对照组) / 对照组 × 100
    • p 值：使用双尾 z 检验或卡方检验
    • 置信区间：差值的 95% CI
    • 统计显著性：p < 0.05 吗？
    • 实际显著性：提升幅度对业务有意义吗？

    如果用户提供了原始数据，生成并运行 Python 脚本来计算上述结果。

  4. 检查护栏指标：

    • 是否有护栏指标（营收、参与度、页面加载时间）出现下降？
    • 主要指标获胜但护栏指标恶化，不一定是真正的胜利

  5. 解读结果：

    结果建议
    显著正向提升，无护栏问题上线 — 全量推送
    显著正向提升，存在护栏顾虑进一步调查 — 理解权衡后再上线
    不显著，但有正向趋势延长测试 — 需要更多数据或更大效应
    不显著，基本持平停止测试 — 未检测到有意义的差异
    显著负向提升不上线 — 回退到对照组，分析原因

  6. 提供分析摘要：

    ## A/B 测试结果：[测试名称]

**假设**：[我们的预期]
**时长**：[X 天] | **样本量**：[N 对照组 / M 实验组]

| 指标 | 对照组 | 实验组 | 提升 | p 值 | 是否显著？ |
|---|---|---|---|---|---|
| [主要指标] | X% | Y% | +Z% | 0.0X | 是/否 |
| [护栏指标] | ... | ... | ... | ... | ... |

**建议**：[上线 / 延长 / 停止 / 进一步调查]
**理由**：[原因]
**后续行动**：[下一步]

逐步思考。保存为 Markdown 格式。如果提供了原始数据，生成 Python 脚本执行计算。

延伸阅读

© killvxk, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

Just SKILL.md in pm-data-analytics/skills/ab-test-analysis of killvxk/pm-skills-zh.

Open the folder on GitHubat commit 5179784

Compare with similar skills

Ab Test Analysis next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Ab Test Analysis compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Ab Test Analysis this skillkillvxk/pm-skills-zh167—~480Automated safety check: PassMIT
Vercel Sandboxvercel/vercel-plugin301—~6.2kAutomated safety check: NotesCustom licence
A/B Test Analysisphuryn/pm-skills27k—~893Automated safety check: PassMIT
AnalyticsNexus-JPF/note-companion8696 repos~2.2kAutomated safety check: PassMIT
Ab Test Setupfreekmurze/dotfiles1k15 repos~1.8kAutomated safety check: PassNone
Ad Test Designeraaron-he-zhu/aaron-marketing-skills2.9k2 repos~2.8kAutomated safety check: PassApache-2.0

Similar skills

  • Vercel Sandbox

    vercel/vercel-plugin

    Official

    Vercel Sandbox guidance — ephemeral Firecracker microVMs for running untrusted code safely.

    301 GitHub stars~6.2k tokensUpdated today
    Marketing & SEOAuto-check: notes

  • A/B Test Analysis

    phuryn/pm-skills

    Validates an experiment's setup, works out lift, p-value and confidence interval from A/B test data, and recommends whether to ship, extend or stop.

    27k GitHub stars~893 tokensUpdated 22 days ago
    Data & AnalyticsAuto-check passed

  • Analytics

    Nexus-JPF/note-companion

    When the user wants to set up, improve, or audit analytics tracking and measurement.

    869 GitHub starsUsed in 6 repos~2.2k tokens
    Marketing & SEOAuto-check passed

  • Ab Test Setup

    freekmurze/dotfiles

    When the user wants to plan, design, or implement an A/B test or experiment.

    1k GitHub starsUsed in 15 repos~1.8k tokens
    Marketing & SEOAuto-check passed

  • Ad Test Designer

    aaron-he-zhu/aaron-marketing-skills

    A skill your agent uses when the user asks to "design an A/B test", "set up a creative/landing test", "run an incrementality test", or "is this result statistically and practically material?"…

    2.9k GitHub starsUsed in 2 repos~2.8k tokens
    Marketing & SEOAuto-check passed

  • Ab Testing

    Cesarjoquin/Marketing-Skills

    When the user wants to plan, design, or implement an A/B test or experiment, or build a growth experimentation program.

    199 GitHub starsUsed in 2 repos~2.8k tokens
    Marketing & SEOAuto-check passed

More from killvxk/pm-skills-zh

All 11 skills in this repo

  • Brainstorm Okrs

    killvxk/pm-skills-zh

    集思广益制定团队级 OKR（目标与关键成果），对齐公司目标——定性目标与可量化关键成果。适用于制定季度 OKR、将团队目标与公司战略对齐、起草目标，或学习如何编写有效的 OKR。

    167 GitHub stars~533 tokensUpdated 6 mo ago
    Auto-check passed

  • Cohort Analysis

    killvxk/pm-skills-zh

    对用户参与度数据执行同期群分析——留存曲线、功能采用趋势及分层洞察。适用于按同期群分析用户留存、研究功能随时间的采用情况、调查流失规律，或识别参与度趋势。

    167 GitHub stars~562 tokensUpdated 6 mo ago
    Auto-check passed

  • Dummy Dataset

    killvxk/pm-skills-zh

    生成用于测试的逼真虚拟数据集，支持自定义列、约束条件及输出格式（CSV、JSON、SQL、Python 脚本）。适用于创建测试数据、构建模拟数据集，或为开发和演示生成示例数据。

    167 GitHub stars~595 tokensUpdated 6 mo ago
    Auto-check passed

  • Interview Script

    killvxk/pm-skills-zh

    创建结构化的用户访谈脚本，包含 JTBD（用户待办任务）探测问题、破冰环节、核心探索及收尾部分。遵循"老妈测试"原则——不引导、不推销，聚焦过去的实际行为。适用于准备用户访谈、创建访谈指南或规划探索性研究时使用。

    167 GitHub stars~547 tokensUpdated 6 mo ago
    Auto-check passed

  • Metrics Dashboard

    killvxk/pm-skills-zh

    定义并设计产品指标看板，包含核心指标、数据来源、可视化类型和告警阈值。适用于创建指标看板、定义 KPI（关键绩效指标）、搭建产品分析体系或制定数据监控计划时使用。

    167 GitHub stars~741 tokensUpdated 6 mo ago
    Auto-check passed

  • Pre Mortem

    killvxk/pm-skills-zh

    对 PRD（产品需求文档）或发布计划进行事前剖析（Pre-mortem）。将风险分类为老虎（真实问题）、纸老虎（被夸大的担忧）和大象（未被说出的隐忧），并按发布阻断、快速跟进或持续跟踪进行分级。适用于发布准备、对产品计划进行压力测试，或识别可能出错的地方。

    167 GitHub stars~475 tokensUpdated 6 mo ago
    Auto-check passed

Works with

Categories

Questions about Ab Test Analysis

What does Ab Test Analysis do?

分析 A/B 测试结果，涵盖统计显著性检验、样本量验证、置信区间计算及上线/延长/停止的决策建议。适用于评估实验结果、判断测试是否达到显著性、解读分流测试数据，或决定是否上线某个实验组。. Ab Test Analysis is an agent skill from killvxk/pm-skills-zh.

When should I use Ab Test Analysis?

Ab Test Analysis fits situations like: tasks that involve A/B testing.

How do I install Ab Test Analysis in Claude Code?

Run `npx skills add killvxk/pm-skills-zh --skill ab-test-analysis -a claude-code`. Or copy the skill folder (pm-data-analytics/skills/ab-test-analysis in killvxk/pm-skills-zh) into .claude/skills/ab-test-analysis in your project. Claude Code loads it when a task matches its description.

How do I install Ab Test Analysis in Codex?

Run `npx skills add killvxk/pm-skills-zh --skill ab-test-analysis -a codex`. Or copy the skill folder (pm-data-analytics/skills/ab-test-analysis in killvxk/pm-skills-zh) into .agents/skills/ab-test-analysis in your project. Codex loads it when a task matches its description.

Can I use Ab Test Analysis in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add killvxk/pm-skills-zh --skill ab-test-analysis -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/ab-test-analysis, .gemini/skills/ab-test-analysis, .github/skills/ab-test-analysis and .opencode/skills/ab-test-analysis in your project.

What does Ab Test Analysis need to run?

SKILL.md names no scripts, command-line tools or credentials: Ab Test Analysis is instructions for the agent only. Our summary lists: Python 3.

Does Ab Test Analysis access the network?

SKILL.md names 1 domain. As links in the text: productcompass.pm. This is read from the text; nothing was executed.

Is Ab Test Analysis safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.

What licence does Ab Test Analysis use?

Ab Test Analysis is published under the MIT licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Ab Test Analysis use?

About 480 tokens (SKILL.md is roughly 1.9k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.

What are the alternatives to Ab Test Analysis?

Skills that share tags, products or a category with Ab Test Analysis: Vercel Sandbox (vercel/vercel-plugin, 301 stars), A/B Test Analysis (phuryn/pm-skills, 27k stars), Analytics (Nexus-JPF/note-companion, 869 stars) and Ab Test Setup (freekmurze/dotfiles, 1k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Ab Test Analysis?

killvxk (a GitHub user) maintains it in killvxk/pm-skills-zh, which has 167 GitHub stars. The repository holds 11 skills in this directory. The repository was last updated on March 16, 2026.

Source: killvxk/pm-skills-zh on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.