Gate an A/B test before a full rollout
Practice the experiment-design order from product-analytics training. The winner is not shipped until the hypothesis, sample-size, and peeking rules are set and the primary metric is significant.
- Write the hypothesis and the single primary metric.
- Compute the required sample size and the test duration.
- Randomize traffic and lock the assignment method.
- Do not peek and stop early unless the pre-registered rule allows it.
- Analyze the primary metric, then look at guards.
- Roll out or abort only after that analysis; do not ship on a dashboard feeling.