跳至正文
OpenAgentSkill
浏览已开放 · 目录记录日期 2026-11-21 · 自动更新和账户服务暂时关闭。
开发与测试

ab-test-setup

Design, analyze, and document A/B tests for conversion, onboarding, pricing, lifecycle, and product experiments. Use when the user asks for `/ab-test-setup`, experiment design, sample size, statistical significance, A/B test analysis, ICE-scored test backlogs, or avoiding common testing mistakes.

eigent-ai · 已有静态检查记录

安装与使用

运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。

npx skills add eigent-ai/agent-skills --skill ab-test-setup

技能说明


name: ab-test-setup description: Design, analyze, and document A/B tests for conversion, onboarding, pricing, lifecycle, and product experiments. Use when the user asks for /ab-test-setup, experiment design, sample size, statistical significance, A/B test analysis, ICE-scored test backlogs, or avoiding common testing mistakes.

A/B Test Setup

Overview

Use this skill to guide the full experiment lifecycle: hypothesis, design, sample size, implementation, analysis, and playbook documentation. Keep tests focused, measurable, and resistant to common errors like peeking early or testing too many changes at once.

Workflow

  1. Define the business goal, user segment, current baseline, target metric, and guardrail metrics.
  2. Write a specific hypothesis:
    • If we change X for audience Y, metric Z will improve because...
  3. Design the test:
    • Control and variant.
    • Primary metric.
    • Secondary and guardrail metrics.
    • Traffic split, eligibility, exclusions, and duration.
  4. Estimate sample size or minimum detectable effect when baseline traffic and conversion rates are available.
  5. Create an implementation checklist:
    • Tracking, randomization, QA, exposure logging, analytics events, and rollback.
  6. Define decision rules before launch:
    • Ship, revert, iterate, or continue testing.
  7. Analyze results after the test reaches the agreed sample size.
  8. Document what changed, what was learned, and follow-up experiments.

Test Backlog Pattern

When building a backlog, score each idea with ICE:

  • Impact: expected business or user benefit.
  • Confidence: evidence quality.
  • Effort: complexity and implementation cost.

Prioritize tests that combine high impact, credible evidence, and low operational risk.

Example Prompts

  • I want to A/B test our signup CTA button. Current conversion rate is 3.2%, 8,000 visitors/month. Help me design the test, calculate the required sample size, and define what success looks like.
  • Our A/B test just hit sample size. Here are the results [paste metrics]. Is this statistically significant? Should we ship the variant, revert, or keep testing?
  • Build a prioritized A/B test backlog for our onboarding flow. Use ICE scoring. Sources to mine: our drop-off analytics, last month's support tickets, and these 3 heatmap observations.