The Bootstrap

The bootstrap is a technique for performing statistical inference. The underlying idea is that most properties of an unknown distribution can be estimated as the same properties of an estimate of that distribution. In most cases, these properties must be estimated by a simulation experiment. The parametric bootstrap can be used when a statistical model is estimated using maximum likelihood since the parameter estimates thus obtained serve to characterise a distribution that can subsequently be used to generate simulated data sets. Simulated test statistics or estimators can then be computed for each of these data sets, and their distribution is an estimate of their distribution under the unknown distribution. The most popular sort of bootstrap is based on resampling the observations of the original data set with replacement in order to constitute simulated data sets, which typically contain some of the original observations more than once, some not at all. A special case of the bootstrap is a Monte Carlo test, whereby the test statistic has the same distribution for all data distributions allowed by the null hypothesis under test. A Monte Carlo test permits exact inference with the probability of Type I error equal to the significance level. More generally, there are two Golden Rules which, when followed, lead to inference that, although not exact, is often a striking improvement on inference based on asymptotic theory. The bootstrap also permits construction of confidence intervals of improved quality. Some techniques are discussed for data that are heteroskedastic, autocorrelated, or clustered.