International Comparison of Public Sector Performance: The Use of Anchoring Vignettes to adjust Self-Reported Data

International comparison of performance has become an influential lever for change in the provision of public services. For health care, patients’ views and opinions are increasingly being recognized as legitimate means for assessing the provision of services, to stimulate quality improvements, and more recently, in evaluating system performance. This has shifted the focus of analyses towards the use of individual-level surveys of performance from the perspective of the user and raises the issue of how to compare appropriately self-reported data across institutional settings and population groups. This represents a major challenge for all public services, the fundamental problem being that comparative evaluation needs to take account of variations in social and cultural expectations and norms when relying on self-reported information. Using data on health systems responsiveness across 18 OECD countries contained within the World Health Survey, this paper outlines the issues that arise in comparative inference that relies on respondent self-reports. The problem of reporting bias is described and illustrated together with potential solutions brought about through the use of anchoring vignettes. The utility of vignettes to aid cross-country analyses and its implications for comparative inference of health system performance are discussed.