A/B Test Calculator

Your details

Analyze mode checks whether a result is statistically significant. Plan mode estimates how many visitors you need before starting.
The probability that a true difference will not be called significant by chance alone. 95% (alpha = 0.05) is the most common threshold.
Two-sided tests for any difference (more conservative). One-sided tests only for an improvement in the expected direction.
Total unique visitors shown the original (control) experience.
Number of desired actions (purchases, sign-ups, clicks) observed in the control group.
Total unique visitors shown the new (variation) experience.
Number of desired actions observed in the variation group.
Relative upliftB wins
0.21%
Z-score2.137
P-value0.0326
Control rate (A)0.04%
Variation rate (B)0.05%
Significant?Yes - B is better at 95% confidence
0.0326 p-value
Very significant<0.01Significant0.01-0.05Marginal0.05-0.1Not significant0.1+

Significant: B lifted conversions by 21.4% relative.

  • Control (A) converted at 4.20%; variation (B) at 5.10%.
  • The z-score is 2.137 and the p-value is 0.0326.
  • The p-value is below 0.05: conventional threshold for significance.

Next stepConsider shipping variation B. Document the test, verify the SRM check, and plan a follow-up experiment.

 = Powered by OnlyCalculators