IDEAS home Printed from https://ideas.repec.org/a/spr/demogr/v52y2015i5p1703-1728.html
   My bibliography  Save this article

Promises and Pitfalls of Anchoring Vignettes in Health Survey Research

Author

Listed:
  • Hanna Grol-Prokopczyk
  • Emese Verdes-Tennant
  • Mary McEniry
  • Márton Ispány

Abstract

Data harmonization is a topic of growing importance to demographers, who increasingly conduct domestic or international comparative research. Many self-reported survey items cannot be directly compared across demographic groups or countries because these groups differ in how they use subjective response categories. Anchoring vignettes, already appearing in numerous surveys worldwide, promise to overcome this problem. However, many anchoring vignettes have not been formally evaluated for adherence to the key measurement assumptions of vignette equivalence and response consistency. This article tests these assumptions in some of the most widely fielded anchoring vignettes in the world: the health vignettes in the World Health Organization (WHO) Study on Global AGEing and Adult Health (SAGE) and World Health Survey (WHS) (representing 10 countries; n=52,388), as well as similar vignettes in the Health and Retirement Study (HRS) (n=4,528). Findings are encouraging regarding adherence to response consistency, but reveal substantial violations of vignette equivalence both cross-nationally and across socioeconomic groups. That is, members of different sociocultural groups appear to interpret vignettes as depicting fundamentally different levels of health. The evaluated anchoring vignettes do not fulfill their promise of providing interpersonally comparable measures of health. Recommendations for improving future implementations of vignettes are discussed. Copyright Population Association of America 2015

Suggested Citation

  • Hanna Grol-Prokopczyk & Emese Verdes-Tennant & Mary McEniry & Márton Ispány, 2015. "Promises and Pitfalls of Anchoring Vignettes in Health Survey Research," Demography, Springer;Population Association of America (PAA), vol. 52(5), pages 1703-1728, October.
    • Handle: RePEc:spr:demogr:v:52:y:2015:i:5:p:1703-1728
    DOI: 10.1007/s13524-015-0422-1
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s13524-015-0422-1
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s13524-015-0422-1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Nigel Rice & Silvana Robone & Peter Smith, 2011. "Analysis of the validity of the vignette approach to correct for heterogeneity in reporting health system responsiveness," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 12(2), pages 141-162, April.
    2. Arie Kapteyn, 2010. "What can we learn from (and about) global aging?," Demography, Springer;Population Association of America (PAA), pages 191-209.
    3. Corrado, L. & Weeks, M., 2010. "Identification Strategies in Survey Response Using Vignettes," Cambridge Working Papers in Economics 1031, Faculty of Economics, University of Cambridge.
    4. Teresa Bago d’Uva & Maarten Lindeboom & Owen O’Donnell & Eddy van Doorslaer, 2011. "Slipping Anchor?: Testing the Vignettes Approach to Identification and Correction of Reporting Heterogeneity," Journal of Human Resources, University of Wisconsin Press, vol. 46(4), pages 875-906.
    5. Teresa Bago d'Uva & Maarten Lindeboom & Owen O'Donnell & Eddy van Doorslaer, 2011. "Education‐related inequity in healthcare with heterogeneous reporting of health," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 174(3), pages 639-664, July.
    6. Kristensen, Nicolai & Johansson, Edvard, 2008. "New evidence on cross-country differences in job satisfaction using anchoring vignettes," Labour Economics, Elsevier, vol. 15(1), pages 96-117, February.
    7. King, Gary & Wand, Jonathan, 2007. "Comparing Incomparable Survey Responses: Evaluating and Selecting Anchoring Vignettes," Political Analysis, Cambridge University Press, vol. 15(1), pages 46-66, January.
    8. Arthur van Soest & Liam Delaney & Colm Harmon & Arie Kapteyn & James P. Smith, 2011. "Validating the use of anchoring vignettes for the correction of response scale differences in subjective questions," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 174(3), pages 575-595, July.
    9. Steven Ruggles, 2014. "Big Microdata for Population Research," Demography, Springer;Population Association of America (PAA), vol. 51(1), pages 287-297, February.
    10. Arie Kapteyn, 2010. "What can we learn from (and about) global aging?," Demography, Springer;Population Association of America (PAA), vol. 47(1), pages 191-209, March.
    11. Burgard, Sarah A. & Chen, Patricia V., 2014. "Challenges of health measurement in studies of health disparities," Social Science & Medicine, Elsevier, vol. 106(C), pages 143-150.
    12. Hao Dong & Cameron Campbell & Satomi Kurosu & Wenshan Yang & James Lee, 2015. "New Sources for Comparative Social Science: Historical Population Panel Data From East Asia," Demography, Springer;Population Association of America (PAA), vol. 52(3), pages 1061-1088, June.
    13. Shetterly, S.M. & Baxter, J. & Mason, L.D. & Hamman, R.F., 1996. "Self-rated health among hispanic vs non-hispanic white adults: The San Luis Valley health and aging study," American Journal of Public Health, American Public Health Association, vol. 86(12), pages 1798-1801.
    14. Arthur van Soest & Hana Vonkova, 2014. "Testing the specification of parametric models by using anchoring vignettes," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 177(1), pages 115-133, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Nicolas R. Ziebarth, 2018. "Social Insurance and Health," Contributions to Economic Analysis, in: Health Econometrics, volume 127, pages 57-84, Emerald Group Publishing Limited.
    2. O'Doherty, M.G. & French, D. & Steptoe, A. & Kee, F., 2017. "Social capital, deprivation and self-rated health: Does reporting heterogeneity play a role? Results from the English Longitudinal Study of Ageing," Social Science & Medicine, Elsevier, vol. 179(C), pages 191-200.
    3. Knott, Rachel J. & Lorgelly, Paula K. & Black, Nicole & Hollingsworth, Bruce, 2017. "Differential item functioning in quality of life measurement: An analysis using anchoring vignettes," Social Science & Medicine, Elsevier, vol. 190(C), pages 247-255.
    4. Coustaury, Camille & Jeannot, Elias & Moreau, Adele & Nietge, Clotilde & Maharani, Asri & Richards, Lindsay & Präg, Patrick, 2023. "Subjective socioeconomic status and self-rated health in the English Longitudinal Study of Aging: A fixed-effects analysis☆☆We thank the anonymous reviewers of Social Science & Medicine for their help," Social Science & Medicine, Elsevier, vol. 336(C).
    5. Präg, Patrick & Mills, Melinda C. & Wittek, Rafael, 2016. "Subjective socioeconomic status and health in cross-national comparison," Social Science & Medicine, Elsevier, vol. 149(C), pages 84-92.
    6. Silvana Robone, 2017. "Comment on ‘Response –scale heterogeneity in the EQ‐5D: Can we use vignettes to address response‐scale heterogeneity in the EQ‐5D? Not if but how’," Health Economics, John Wiley & Sons, Ltd., vol. 26(3), pages 395-397, March.
    7. Santosh Jatrana, 2021. "Gender differences in self-reported health and psychological distress among New Zealand adults," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 45(21), pages 693-726.
    8. Molina, Teresa, 2017. "Adjusting for heterogeneous response thresholds in cross-country comparisons of self-reported health," The Journal of the Economics of Ageing, Elsevier, vol. 10(C), pages 1-20.
    9. Oksuzyan, Anna & Dańko, Maciej J. & Caputo, Jennifer & Jasilionis, Domantas & Shkolnikov, Vladimir M., 2019. "Is the story about sensitive women and stoical men true? Gender differences in health after adjustment for reporting behavior," Social Science & Medicine, Elsevier, vol. 228(C), pages 41-50.
    10. Laura Rossouw & Teresa Bago d’Uva & Eddy Doorslaer, 2018. "Poor Health Reporting? Using Anchoring Vignettes to Uncover Health Disparities by Wealth and Race," Demography, Springer;Population Association of America (PAA), vol. 55(5), pages 1935-1956, October.
    11. Teresa Molina, 2016. "Reporting Heterogeneity and Health Disparities Across Gender and Education Levels: Evidence From Four Countries," Demography, Springer;Population Association of America (PAA), vol. 53(2), pages 295-323, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hanna Grol-Prokopczyk, 2018. "In Pursuit of Anchoring Vignettes That Work: Evaluating Generality Versus Specificity in Vignette Texts," The Journals of Gerontology: Series B, The Gerontological Society of America, vol. 73(1), pages 54-63.
    2. William H. Greene & Mark N. Harris & Rachel J. Knott & Nigel Rice, 2021. "Specification and testing of hierarchical ordered response models with anchoring vignettes," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(1), pages 31-64, January.
    3. Raskina, Yulia & Podkorytova, Olga & Kuchakov, Ruslan, 2022. "Health determinants and the reporting heterogeneity bias in Russia: Anchoring vignettes approach," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 66, pages 118-143.
    4. Bertoni, Marco, 2015. "Hungry today, unhappy tomorrow? Childhood hunger and subjective wellbeing later in life," Journal of Health Economics, Elsevier, vol. 40(C), pages 40-53.
    5. Andrew M. Jones; Nigel Rice, Silvana Robone; & Nigel Rice; & Silvana Robone:, 2012. "A comparison of parametric and non-parametric adjustments using vignettes for self-reported data," Health, Econometrics and Data Group (HEDG) Working Papers 12/10, HEDG, c/o Department of Economics, University of York.
    6. Zhang, Hao & Bago d’Uva, Teresa & van Doorslaer, Eddy, 2015. "The gender health gap in China: A decomposition analysis," Economics & Human Biology, Elsevier, vol. 18(C), pages 13-26.
    7. Rachel J. Knott & Nicole Black & Bruce Hollingsworth & Paula K. Lorgelly, 2017. "Response‐Scale Heterogeneity in the EQ‐5D," Health Economics, John Wiley & Sons, Ltd., vol. 26(3), pages 387-394, March.
    8. Laura Rossouw & Teresa Bago d’Uva & Eddy Doorslaer, 2018. "Poor Health Reporting? Using Anchoring Vignettes to Uncover Health Disparities by Wealth and Race," Demography, Springer;Population Association of America (PAA), vol. 55(5), pages 1935-1956, October.
    9. Arthur van Soest & Hana Vonkova, 2014. "Testing the specification of parametric models by using anchoring vignettes," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 177(1), pages 115-133, January.
    10. Omar Paccagnella, 2011. "Anchoring vignettes with sample selection due to non‐response," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 174(3), pages 665-687, July.
    11. Knott, R. & Lorgelly, P. & Black, N. & Hollingsworth, B., 2016. "Differential item functioning in the EQ-5D: An exploratory analysis using anchoring vignettes," Health, Econometrics and Data Group (HEDG) Working Papers 16/14, HEDG, c/o Department of Economics, University of York.
    12. Franco Peracchi & Claudio Rossetti, 2013. "The heterogeneous thresholds ordered response model: identification and inference," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 176(3), pages 703-722, June.
    13. Teresa Bago d’Uva & Maarten Lindeboom & Owen O’Donnell & Eddy van Doorslaer, 2011. "Slipping Anchor?: Testing the Vignettes Approach to Identification and Correction of Reporting Heterogeneity," Journal of Human Resources, University of Wisconsin Press, vol. 46(4), pages 875-906.
    14. Corrado, L. & Weeks, M., 2010. "Identification Strategies in Survey Response Using Vignettes," Cambridge Working Papers in Economics 1031, Faculty of Economics, University of Cambridge.
    15. O'Doherty, M.G. & French, D. & Steptoe, A. & Kee, F., 2017. "Social capital, deprivation and self-rated health: Does reporting heterogeneity play a role? Results from the English Longitudinal Study of Ageing," Social Science & Medicine, Elsevier, vol. 179(C), pages 191-200.
    16. Knott, Rachel J. & Lorgelly, Paula K. & Black, Nicole & Hollingsworth, Bruce, 2017. "Differential item functioning in quality of life measurement: An analysis using anchoring vignettes," Social Science & Medicine, Elsevier, vol. 190(C), pages 247-255.
    17. Zhiyong Huang & Haoxian Wang & Wenyuan Zheng, 2021. "An extended hierarchical ordered probit model robust to heteroskedastic vignette perceptions with an application to functional limitation assessment," PLOS ONE, Public Library of Science, vol. 16(3), pages 1-19, March.
    18. Kaiser, Caspar, 2022. "Using memories to assess the intrapersonal comparability of wellbeing reports," Journal of Economic Behavior & Organization, Elsevier, vol. 193(C), pages 410-442.
    19. Nigel Rice & Silvana Robone & Peter Smith, 2011. "Analysis of the validity of the vignette approach to correct for heterogeneity in reporting health system responsiveness," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 12(2), pages 141-162, April.
    20. Daniel J. Benjamin & Kristen Cooper & Ori Heffetz & Miles S. Kimball, 2023. "From Happiness Data to Economic Conclusions," NBER Working Papers 31727, National Bureau of Economic Research, Inc.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:demogr:v:52:y:2015:i:5:p:1703-1728. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.