A/B Test Significance Calculator
Compare conversions of two variants and see the uplift, p-value, confidence interval and whether the difference is statistically significant.
How it works
The calculator runs a two-proportion z-test. It compares the conversion rates of A and B and gives the p-value – the probability of seeing a difference at least this large if both variants were really the same. If the p-value is below 1 − confidence (0.05 at 95%), the result is significant.
Example: 420 conversions from 10,000 visitors for A (4.20%) and 490 from 10,000 for B (4.90%) is an uplift of +16.67% with p = 0.0175 – significant at 95%.
Confidence interval
The interval shows the range of plausible differences between B and A. If it does not include zero, the result is significant; a wide interval means the true effect could be small.
Traffic split check
If a 50/50 test shows clearly unequal visitor numbers, something in the setup is wrong – a sample ratio mismatch (SRM). Such a result is not trustworthy, whatever the p-value says.
Do not stop early
Checking results every day and stopping when they turn significant greatly increases false positives. Decide the sample size first and run the test until you reach it.
Frequently asked questions
How do I know if my A/B test is significant?
Enter visitors and conversions for both variants; if the p-value is below 0.05, the result is significant at 95% confidence.
What is a p-value in an A/B test?
The probability of seeing a difference at least this large if the variants really performed the same.
Should I use a one-sided or two-sided test?
Two-sided by default – it also detects when B is worse; one-sided only if you decided that before the test.