The Effect of Low Survey Response Rates on Estimates of Alcohol Consumption in a General Population Survey

Background: Response rates for surveys of alcohol use are declining for all modes of administration (postal, telephone, face-to-face). Low response rates may result in estimates that are biased by selective non-response. We examined non-response bias in the NZ GENACIS survey, a postal survey of a random electoral roll sample, with a response rate of 49.5% (n = 1924). Our aim was to estimate the magnitude of non-response bias in estimating the prevalence of current drinking and heavy episodic (binge) drinking. Methods: We used the “continuum of resistance” model to guide the investigation. In this model the likelihood of response by sample members is related to the amount of effort required from the researchers to elicit a response. First, the demographic characteristics of respondents and non-respondents were compared. Second, respondents who returned their questionnaire before the first reminder (early), before the second reminder (intermediate) or after the second reminder (late) were compared by demographic characteristics, 12-month prevalence of drinking and prevalence of binge drinking. Results: Demographic characteristics and prevalence of binge drinking were significantly different between late respondents and early/intermediate respondents, with the demographics of early and intermediate respondents being similar to people who refused to participate while late respondents were similar to all other non-respondents. Assuming non-respondents who did not actively refuse to participate had the same drinking patterns as late respondents, the prevalence of binge drinking amongst current drinkers was underestimated. Adjusting the prevalence of binge drinkers amongst current drinkers using population weights showed that this method of adjustment still resulted in an underestimate of the prevalence. Conclusions: The findings suggest non-respondents who did not actively refuse to participate are likely to have similar or more extreme drinking behaviours than late respondents, and that surveys of health compromising behaviours such as alcohol use are likely to underestimate the prevalence of these behaviours.