Indices of non‐ignorable selection bias for proportions estimated from non‐probability samples

Rising costs of survey data collection and declining response rates have caused researchers to turn to non‐probability samples to make descriptive statements about populations. However, unlike probability samples, non‐probability samples may produce severely biased descriptive estimates due to selection bias. The paper develops and evaluates a simple model‐based index of the potential selection bias in estimates of population proportions due to non‐ignorable selection mechanisms. The index depends on an inestimable parameter ranging from 0 to 1 that captures the amount of deviation from selection at random and is thus well suited to a sensitivity analysis. We describe modified maximum likelihood and Bayesian estimation approaches and provide new and easy‐to‐use R functions for their implementation. We use simulation studies to evaluate the ability of the proposed index to reflect selection bias in non‐probability samples and show how the index outperforms a previously proposed index that relies on an underlying normality assumption. We demonstrate the use of the index in practice with real data from the National Survey of Family Growth.