Statistical Power Calculator
Estimate statistical power and required sample size for a two-proportion test.
How statistical power works
Statistical power is the probability that a test will detect an effect when one truly exists: Power = 1 − β, where β is the probability of a false negative (Type II error).
Power depends on the effect size, the significance level α, the sample size, and the variability of the data. Given any four of these, the fifth can usually be found — this calculator finds power or the required sample size.
V1 supports one clearly defined, common scenario: the two-proportion z-test (e.g., comparing conversion rates in an A/B test) using the standard normal approximation. Other models are not faked — only this scenario is implemented and documented.
The formulas
Power = Φ( δ/SE₁ − z_{1−α/2}·SE₀/SE₁ ) · Power = 1 − β
For two proportions, δ = |p₁−p₂|, SE₀ uses the pooled proportion under the null, and SE₁ uses the two separate proportions. The required per-group sample size is the closed-form solution for a target power. Cohen's h = 2·arcsin(√p₂) − 2·arcsin(√p₁) is reported as the standardized effect size.
Worked example
Baseline 50% → target 60%, α = 0.05
With 200 subjects per group, power is about 0.52 — only slightly better than a coin flip. To reach 80% power for this effect you would need about 388 subjects per group.
Frequently asked questions
What is the difference between α, β and power?
α (alpha) is the significance level — the accepted probability of a false positive. β (beta) is the probability of a false negative. Power = 1 − β is the probability of detecting a real effect.
What is an effect size?
An effect size is a standardized magnitude of the difference you want to detect. Here it is the difference between two proportions, also reported as Cohen's h.
What scenario does this calculator support?
V1 supports the two-proportion z-test (two independent groups with binary outcomes, e.g. A/B conversion rates). Power for other models — t-tests, ANOVA — is not implemented and is not silently approximated.
Related research methods
- Hypothesis Testing
- Sample Size Determination
When to use this tool
Use this when planning a study to check whether you can detect the effect you care about with a realistic sample, or when reviewing whether an underpowered study could have missed a real effect.
Assumptions
- A specified significance level (α).
- A target effect size — the smaller the effect you want to detect, the larger the sample you need.
- A target power (commonly 0.80).
Limitations
- Power analysis requires you to specify an effect size, which is often uncertain before data collection.
- Low power does not prove an effect is absent — it means the study was unlikely to detect it.
How to interpret the result
Power is the probability of rejecting the null hypothesis when a true effect of the specified size exists. Power of 0.80 means an 80% chance of detecting that effect.
How to report the result
Report the significance level, target power, assumed effect size, and the resulting (or achieved) sample size.
Next steps in your analysis
Related tools
Related guides
- How to Choose the Right Statistical TestA practical framework for picking the right statistical test: start from your research question, data type and study design, then check the assumptions.
- T-Test vs Z-Test vs ANOVAWhen to use a z-test, a t-test or an ANOVA, the assumptions behind each, and how the tests relate — including why F equals t² for two groups.
- Effect Size and Statistical Power: Designing Studies That Can Detect EffectsWhy statistical significance is not enough — how to measure effect size with Cohen's d and design studies with adequate statistical power before you collect data.
Built by PanelRoster
PanelRoster is an enterprise operations management platform for organizations coordinating distributed teams, assignments, workflows, quality assurance and operational execution.
Explore PanelRoster