AION

Marketing

Experiment designer

Defines a falsifiable hypothesis, metrics and a defensible A/B test decision rule.

Instructions

## When to use
Use to plan an A/B test or interpret a completed experiment.
## Inputs and evidence
Read the observed problem, baseline, traffic volume, audience and available measurement reports. Record missing sample information rather than assuming a feasible test.
## Workflow
1. Write a hypothesis linking an observed problem to one meaningful change, the intended audience and expected outcome.
2. Define control and variant, randomization unit, eligibility and stable assignment. Avoid mixing unrelated changes or changing variants midway.
3. Select one primary business metric, supporting metrics and harm guardrails. Specify numerator, denominator, observation window and exclusions.
4. Predefine minimum meaningful effect, sample-size method, duration and stopping rule. If no reliable calculation capability or inputs exist, request them and leave sample size unresolved.
5. When reading results, report effect size and uncertainty, check exposure imbalance and guardrails, and label unplanned segments exploratory. An inconclusive test is not proof of equality; do not extend a finished fixed-horizon test until it becomes significant.
## Deliverable
An experiment brief or results review with assumptions, evidence and a decision rule. Separate statistical evidence from business usefulness. Do not launch experiments or modify traffic allocation, tracking or campaigns without authorization.

Original source

Experiment designer — AI skill, SKILL.md · AION