跳至正文
OpenAgentSkill
浏览已开放 · 目录记录日期 2026-11-21 · 自动更新和账户服务暂时关闭。
开发与测试

ad-test-designer

Use when the user asks to "design an A/B test", "set up a creative/landing test", "run an incrementality test", or "is this result statistically and practically material?"; produces a hypothesis, variant matrix, sample-size/duration/power plan, and a documented effect/uncertainty read from own exported results. It applies only a precommitted owner-approved action rule; the statistical helper never chooses a business action. Not for producing variants — use ad-creative-builder; not for reading back one shipped change — use paid-measurement-loop. 广告AB测试设计/实验设计/显著性判定/增效测试

aaron-he-zhu · 已有 AI 审核记录

安装与使用

运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。

npx skills add aaron-he-zhu/aaron-marketing-skills --skill ad-test-designer

技能说明


name: ad-test-designer slug: aaron-ad-test-designer displayName: "Ad Test Designer · 广告AB测试设计" summary: "广告AB测试设计/实验设计/显著性判定/增效测试" description: 'Use when the user asks to "design an A/B test", "set up a creative/landing test", "run an incrementality test", or "is this result statistically and practically material?"; produces a hypothesis, variant matrix, sample-size/duration/power plan, and a documented effect/uncertainty read from own exported results. It applies only a precommitted owner-approved action rule; the statistical helper never chooses a business action. Not for producing variants — use ad-creative-builder; not for reading back one shipped change — use paid-measurement-loop. 广告AB测试设计/实验设计/显著性判定/增效测试' version: "20.1.0" license: Apache-2.0 compatibility: "Claude Code and compatible agent-skill hosts" homepage: "https://github.com/aaron-he-zhu/aaron-marketing-skills" when_to_use: "Use when designing a creative/landing A/B/n or incrementality test, or when reading effect size, uncertainty, and guardrails from a finished own-data test. Apply a business action only when its owner and decision rule were precommitted; otherwise return decision UNDECIDED. Not for generating variants (use ad-creative-builder) or reading back one already-shipped change (use paid-measurement-loop)." argument-hint: "<what to test / results CSV> [profile: direct-response|prospecting|incremental-profit] [baseline] [alpha/power/MDE]" metadata: {"author": "aaron-he-zhu", "version": "20.1.0", "discipline": "ad", "phase": "orchestrate", "geo-relevance": "low", "hermes": {"tags": ["marketing", "ad", "orchestrate"], "category": "ad"}, "openclaw": {"emoji": "🎯", "homepage": "https://github.com/aaron-he-zhu/aaron-marketing-skills"}}

Ad Test Designer

Designs paid-ad creative/landing A/B/n and incrementality tests and reads them out: hypothesis, variant matrix, sample-size/duration/power plan, effect size, uncertainty, practical-effect status, and guardrail state. This skill owns experiment design + statistical interpretation. It may apply an owner-approved, precommitted action rule, but it never treats a p-value or helper output as an automatic business decision. It does not produce variants (ad-creative-builder), read back one already-shipped change (paid-measurement-loop), or do cross-channel reporting (performance-analyzer).

Quick Start

Design an A/B test for two landing-page hero variants. Baseline CVR is 3%, I want to detect a 15% lift. Goal is DR.
I have 4 RSA creative variants to test on a prospecting set. Build the variant matrix, sample size, and run duration.
Here's my finished test results CSV (variant, sessions, conversions). Is the winner significant — promote or kill?

Skill Contract

  • Expected output: a test design (hypothesis, variant matrix, immutable test/variant/measurement binding, primary/secondary/guardrail metrics, sample-size + duration + power plan) and/or a read-out bound to that exact design (effect estimate, interval, statistical flag, practical-effect flag, guardrails, and either an owner-governed recommendation or decision: UNDECIDED).
  • Reads: what the user wants to test, the ROAS profile (direct-response|prospecting|incremental-profit), baseline CVR/CTR and traffic volume, stable control/candidate refs, the exact creative or landing artifact hash, and the measurement-contract ref/hash; for a read-out, the user's own exported results CSV (variant, sessions/impressions, conversions/clicks) plus the original binding.
  • Writes: a user-facing test-design or read-out doc plus a ### Handoff Summary.
  • Promotes: the chosen hypothesis, design parameters, calculated read-out, and any explicitly owner-approved action (ask before writing memory).
  • Done when: a falsifiable hypothesis is stated; the matrix isolates one variable per variant; the control, candidate, variant hash, signal spec, and measurement contract are bound; baseline, MDE, alpha, power, multiplicity/sequential policy, duration, and guardrails are declared; and a read-out reports effect/interval/statistical/practical flags with Calculated provenance against the same binding. A mismatch returns NEEDS_INPUT/UNDECIDED; without a precommitted action rule and owner, return decision: UNDECIDED.
  • Primary next skill: ad-creative-builder (to produce the winning direction) or paid-measurement-loop.

Handoff Summary

Emit the standard shape from skill-contract.md §Handoff Summary Format.

Data Sources

See CONNECTORS.md for tool category placeholders. Every input is the user's own data, manually exported. Keyed ad-platform APIs (Google Ads SDK, Meta Marketing API) are an optional Tier-2/3 MCP convenience — never required to design a test or read one out.

Statistical facts (keyless): python3 "${CLAUDE_PLUGIN_ROOT}/scripts/connectors/experiment.py" proportion --control <conv> <n> --variant <conv> <n> --alpha <alpha> --min-lift <relative-bar> returns rates, effect size, intervals, p-value, and separate statistical/practical flags. Revenue/AOV-style samples use continuous; prospective sizing uses samplesize. Every derived value is Calculated; the helper deliberately returns no winner, promote, rollback, or kill action.

NeedSource export (own data)Category
Baseline CVR/CTR, traffic volumecampaign report~~ad platform
Test results (variant, sessions, conversions)experiment/results CSV export~~ad platform, ~~web analytics
Conversion truth set for the read-outGA4 / ecommerce export~~web analytics, ~~ecommerce

With manual data only: for a design, ask for the baseline CVR/CTR, traffic/day, and the minimum lift worth detecting. For a read-out, ask for the results CSV wit

阅读完整来源 ↗