Gate an A/B test before a full rollout

Practice the experiment-design order from product-analytics training. The winner is not shipped until the hypothesis, sample-size, and peeking rules are set and the primary metric is significant.

  1. Write the hypothesis and the single primary metric.
  2. Compute the required sample size and the test duration.
  3. Randomize traffic and lock the assignment method.
  4. Do not peek and stop early unless the pre-registered rule allows it.
  5. Analyze the primary metric, then look at guards.
  6. Roll out or abort only after that analysis; do not ship on a dashboard feeling.