Bootstrap method

The bootstrap method estimates a statistic’s sampling uncertainty by repeatedly resampling observed data and recalculating the statistic.

Empirical distribution: A probability distribution that assigns observed data points their relative frequencies. The bootstrap treats the observed data’s empirical distribution as an estimate of the population.
Sampling with replacement: A sampling scheme in which each selected item is returned before the next draw. Replacement allows bootstrap samples to contain duplicates and omit observed cases.
Nonparametric bootstrap: A bootstrap that resamples observed data without specifying a parametric population distribution. It is the standard form when the empirical distribution serves as the population estimate.
Jackknife resampling: A resampling method that recalculates a statistic after systematically omitting one observation at a time. The jackknife uses structured omissions, unlike the bootstrap’s random draws with replacement.
Bradley Efron: An American statistician who introduced the modern bootstrap method in a 1979 paper. Efron named and formalized the method that made resampling a general tool for statistical inference.
Bootstrap sample: A sample drawn with replacement from the observed dataset, usually with the original sample size. Each replicate begins by drawing one of these resamples.
Statistic: A quantity calculated from sample data, such as a mean, median, or regression coefficient. The bootstrap estimates uncertainty in a chosen statistic by recalculating it on every resample.
Parametric bootstrap: A bootstrap that simulates new datasets from a fitted parametric probability model. It replaces empirical resampling with draws from an explicitly modeled distribution.
Permutation test: A hypothesis test that evaluates a statistic under rearrangements allowed by a null hypothesis. Permutation tests construct a null distribution, while the bootstrap estimates sampling variability.
Jackknife (statistics): A resampling technique that estimates bias and variance by leaving out observations in turn. The jackknife was an established predecessor to Efron’s more general bootstrap approach.

Linked from 11 pages