create-experiment
Guide the design and creation of a new FME experiment (fme_experiment): inventories existing metrics and event health, hands off gap-filling to create-metric/instrument-metric and primary/secondary selection to choose-metric, defines baseline vs. variant treatments against the flag's live definition, drafts a hypothesis, then creates the experiment. Doesn't explain an experiment already running or its results - see review-experiment-results. Doesn't create the underlying flag or its treatments - see manage-feature-flags. Use when asked to design or set up an A/B test. Trigger phrases: create an experiment, set up an A/B test, design an experiment, launch a test on this flag, new experiment for X.
Harness · 已有静态检查记录
安装与使用
运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。
npx skills add harness/harness-skills --skill create-experiment技能说明
name: create-experiment description: >- Guide the design and creation of a new FME experiment (fme_experiment): inventories existing metrics and event health, hands off gap-filling to create-metric/instrument-metric and primary/secondary selection to choose-metric, defines baseline vs. variant treatments against the flag's live definition, drafts a hypothesis, then creates the experiment. Doesn't explain an experiment already running or its results - see review-experiment-results. Doesn't create the underlying flag or its treatments - see manage-feature-flags. Use when asked to design or set up an A/B test. Trigger phrases: create an experiment, set up an A/B test, design an experiment, launch a test on this flag, new experiment for X. metadata: author: Harness version: 1.1.0 mcp-server: harness-mcp-v2 license: Apache-2.0 compatibility: >- Requires Harness MCP v2 server (harness-mcp-v2) with fme_experiment (create/get), fme_feature_flag / fme_feature_flag_definition, fme_environment, fme_metric, and fme_event_type. fme_experiment is Harness-native only (org_id + project_id) - there is no legacy workspace_id support, unlike fme_feature_flag. create requires environment_id as a param; the parent (Feature Flag or AI Config) must already exist in that environment.
Create Experiment
Guide a user through designing a well-formed FME experiment end to end - metrics, baseline vs. variant, hypothesis - then create it. This skill orchestrates the decisions; it delegates the decisions other skills already own rather than re-deriving them.
For every step marked Stop condition, read references/stop-conditions.md
before improvising.
Prerequisites
Establish org_id + project_id. Resolve which environment the experiment
runs in before anything else - fme_experiment create needs environment_id
as a param, and the flag's treatments (Step 2) are also read per environment:
Call MCP tool: harness_list
Parameters:
resource_type: "fme_environment"
org_id: "<org_id>"
project_id: "<project_id>"
Stop condition if the user hasn't said which environment, ask - don't default to the first one returned or to production.
Instructions
Step 1: Resolve the parent (flag or AI Config)
Call MCP tool: harness_list
Parameters:
resource_type: "fme_feature_flag"
org_id: "<org_id>"
project_id: "<project_id>"
filters: { name: "<flag name>" }
An experiment's parent is {type: "FEATURE_FLAG"|"AI_CONFIG", id?, name?} -
CONFIG parents 404 on this endpoint. Default to FEATURE_FLAG unless the
user says AI Config. If the flag doesn't exist yet, this skill doesn't create
it - point to /manage-feature-flags first, then come back.
Step 2: Confirm baseline vs. variant against the flag's live treatments
The experiment's baselineTreatment/comparisonTreatments must be real
treatment names already configured on the flag in the chosen environment, not
invented labels like "control"/"treatment" (control is reserved for "the
SDK couldn't evaluate the flag" and is never a configured treatment):
Call MCP tool: harness_get
Parameters:
resource_type: "fme_feature_flag_definition"
org_id: "<org_id>"
project_id: "<project_id>"
params: { feature_flag_name: "<flag_name>", environment_id: "<environment_id>" }
Read treatments[].name. A 404 here means the flag has no definition in that
environment; if every environment 404s, the flag was created but never
configured anywhere, so there are no treatments to compare and this skill
can't proceed - say so and point to /manage-feature-flags to configure the
flag first. Use the definition's baselineTreatment as the default answer
for the baseline, but still confirm - the experiment's own
baselineTreatment is a separate field and can disagree with the
definition's. Don't use defaultTreatment for this: it's what killed traffic
and traffic outside trafficAllocation receive, unrelated to the baseline.
Also read trafficType.id (Step 4 filters metrics by it) and check that the
treatments will actually receive traffic in this environment:
isKilledisfalse- while killed, everyone getsdefaultTreatment.trafficAllocation> 0.- The baseline and every comparison treatment have
size> 0 in thedefaultRulebuckets, or in thebucketsof the targeting rule the experiment will use. Ifrulesis non-empty, ask which one applies - it becomes Step 6'srule.
If any check fails, the experiment collects nothing for that treatment until
the definition changes. Say which check failed and ask whether to continue
(e.g. the user plans to ramp traffic before startAt); fixing it is a
flag-definition change outside this skill.
Then check for an experiment already running on this flag and environment:
Call MCP tool: harness_list
Parameters:
resource_type: "fme_experiment"
org_id: "<org_id>"
project_id: "<project_id>"
filters:
parent_type: "FEATURE_FLAG"
parent_name: "<flag_name>"
environment_id: "<environment_id>"
status: ["ACTIVE"]
If one exists, show it (name, startAt/endAt, treatments) and ask whether
the user still wants a second experiment before continuing.
Stop condition if there are fewer than two treatments, or it's unclear
which existing treatment is the baseline vs. which is the variant under
test - see references/stop-conditions.md. This skill does not add
treatments to a flag; that's a flag-definition change outside this skill's
scope.
Step 3: Write the hypothesis
Get a hypothesis in the same form /choose-metric requires before
recommending a primary metric: "if [change], then [metric] will
increase/decrease because [reasoning]". hypothesis is a free-text field on
fme_experiment (max 500 chars) - store the causal reasoning here, not just
the metric name.
Stop condition if the hypothesis is missing, or states an outcome without a causal reason ("conversion will go up" w
阅读完整来源 ↗