A/B Test Significance Calculator
Calculate the statistical significance of an A/B test by comparing the conversion rates of a control group and a variant group using a two-proportion z-test.
About the A/B Test Significance Calculator
A/B testing compares two versions of something — a webpage, an email, a checkout flow — by measuring the conversion rate of a control group (A) against a variant group (B). This calculator runs a two-proportion z-test to determine whether an observed difference is likely real or just random noise.
Formula: Pooled rate p̄ = (Conversions A + Conversions B) ÷ (Visitors A + Visitors B) · SE = √(p̄(1 − p̄)(1/nᵌ + 1/nᵍ)) · z = (Rate B − Rate A) ÷ SE, converted to a two-tailed p-value using the standard normal distribution
A p-value below 0.05 is conventionally treated as statistically significant, meaning the difference between groups is unlikely to be due to chance alone at the 95% confidence level. Relative uplift shows the percentage change in conversion rate that the variant achieved over the control. As with any significance test, larger sample sizes make it easier to detect small but real differences, so an insignificant result with a small sample doesn't necessarily mean there's no effect.