Bootstrapping

Bootstrapping estimates how much a statistic, such as a mean, median, or ratio, would vary across different samples, without relying on a formula or assuming the data follows a particular distribution. It does this by treating the observed sample as a stand-in for the population and drawing many new samples from it, with replacement, each the same size as the original.

For each resample, the statistic of interest is recalculated; repeating this thousands of times builds up a distribution of the statistic that can be used directly to estimate a standard error or a confidence interval, simply by taking percentiles of the resampled values. This makes bootstrapping especially useful for statistics like the median or a ratio of two metrics, where a clean analytical formula for the standard error is difficult or does not exist.

Bootstrapping matters because it works well with real-world, non-normal, or complex data where classical formulas' assumptions do not hold, and it requires no more than a computer and the original sample. It is closely related to Monte Carlo simulation, though bootstrapping resamples from actual observed data rather than a theoretical model. The main pitfall is applying it to a very small original sample, which limits how well the resamples represent the true population, or to data with strong dependence between observations, like time series, which the basic method assumes away.