Tool · Experimentation

Will this test actually tell you anything?

Size an A/B test before it starts, then see what happens when someone checks it every day and stops at the first significant result.

Sample size per group by the lift you want to detect

How often a test with no real effect declares a winner

Each extra look is another chance for noise to cross the line. The fix is to decide the sample size first, or use a sequential method built for repeated looks.