Experiences and lessons learned from bootstrapping random-effects predictions

Background: Random effects are commonly modeled in multilevel, longitudinal, and latent-variable settings. Rather than estimating fixed effects for specific clusters of data, "predictions" can be made as the mode or mean of posterior distributions that arise as the product of the random effect (an empirical Bayes prior) and the likelihood function conditional on cluster membership. Analyses and data: This presentation will explore the experiences and lessons learned in using the bootstrap for inference on random-effects predictors following logistic regression models conducted through both xtmelogit and gllamm. In the United Kingdom, 203 hospitals were compared on the quality of care received by 10,617 stroke patients through multilevel logistic regression models. Results and considerations: Multilevel modeling and prediction are both computer-intensive, and so bootstrapping them is especially time-consuming. Examples from do-files with some helpful approaches will be shown. A small proportion of modal best linear unbiased predictors contained errors, possibly arising from the prediction algorithm. Various bootstrap confidence intervals exhibited problems such as excluding the point prediction and degeneracy. Methods for tracing the source will be presented. Conclusion: Bootstrapping provides flexible but time-consuming inference for individual clusters' predictions. However, there are potential problems that analysts should be aware of.