Effect size

Effect size measures how large a difference or relationship actually is, separate from whether it is statistically detectable. Two studies can report the same p-value with very different effect sizes, because a p-value is driven jointly by the size of the effect and the sample size, a tiny effect can be "significant" if the sample is large enough.

Common effect-size measures include Cohen's d, which expresses the difference between two group means in standard deviation units, and correlation-based or percentage-based measures for other kinds of comparisons. Unlike a raw difference in the original units, a standardized effect size can be compared across studies, variables, and metrics that use different scales.

Effect size matters because it answers the practical question a p-value cannot: is this difference big enough to matter? A statistically significant result with a negligible effect size may not justify a product change, while a promising but non-significant result with a moderate effect size may simply need a larger sample to confirm. Effect size is also a required input to a statistical power calculation before running an experiment, since power depends directly on how large an effect the study is designed to detect, see minimum detectable effect.