IDEAS home Printed from https://ideas.repec.org/a/eee/socmed/v190y2017icp247-255.html
   My bibliography  Save this article

Differential item functioning in quality of life measurement: An analysis using anchoring vignettes

Author

Listed:
  • Knott, Rachel J.
  • Lorgelly, Paula K.
  • Black, Nicole
  • Hollingsworth, Bruce

Abstract

Systematic differences in the ways that people use and interpret response categories (differential item functioning, DIF) can introduce bias when using self-assessments to compare health or quality of life across heterogeneous groups. This paper reports on an exploratory analysis involving the use of anchoring vignettes to identify DIF in a commonly used measure for assessing health-related quality of life - namely the EQ-5D. Using data from a bespoke (i.e. custom) survey that recruited a representative sample of 4300 respondents from the general Australian population in 2014 and 2015, we find that the assumptions of response consistency (RC) and vignette equivalence (VE) hold in a sub-sample of respondents aged 55–65 years (n = 914), which demonstrates that vignettes can appropriately identify DIF in EQ-5D reporting for this age group. We find that the EQ-5D is indeed subject to DIF, and that failure to account for DIF can lead to conclusions that are misleading when using the instrument to compare health or quality of life across heterogeneous groups. We also provide several important insights in terms of the identifying assumptions of RC and VE. We conclude that the implications of DIF could be of considerable importance, not only for outcomes research, but for funding decisions in healthcare more broadly given the strong reliance on patient-reported outcome measures in economic evaluations for health technology assessment.

Suggested Citation

  • Knott, Rachel J. & Lorgelly, Paula K. & Black, Nicole & Hollingsworth, Bruce, 2017. "Differential item functioning in quality of life measurement: An analysis using anchoring vignettes," Social Science & Medicine, Elsevier, vol. 190(C), pages 247-255.
    • Handle: RePEc:eee:socmed:v:190:y:2017:i:c:p:247-255
    DOI: 10.1016/j.socscimed.2017.08.033
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0277953617305142
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.socscimed.2017.08.033?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Patrick W. Sullivan & Vahram Ghushchyan, 2006. "Preference-Based EQ-5D Index Scores for Chronic Conditions in the United States," Medical Decision Making, , vol. 26(4), pages 410-420, July.
    2. Franco Peracchi & Claudio Rossetti, 2013. "The heterogeneous thresholds ordered response model: identification and inference," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 176(3), pages 703-722, June.
    3. Arie Kapteyn & James P. Smith & Arthur Van Soest, 2013. "Are A mericans Really Less Happy with Their Incomes?," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 59(1), pages 44-65, March.
    4. Hendrik Jürges & Joachim Winter, 2013. "Are Anchoring Vignettes Ratings Sensitive To Vignette Age And Sex?," Health Economics, John Wiley & Sons, Ltd., vol. 22(1), pages 1-13, January.
    5. Arie Kapteyn & James P. Smith & Arthur Van Soest, 2013. "Are A mericans Really Less Happy with Their Incomes?," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 59(1), pages 44-65, March.
    6. repec:cup:apsrev:v:98:y:2004:i:01:p:191-207_00 is not listed on IDEAS
    7. Viola Angelini & Danilo Cavapozzi & Luca Corazzini & Omar Paccagnella, 2014. "Do Danes and Italians Rate Life Satisfaction in the Same Way? Using Vignettes to Correct for Individual-Specific Scale Biases," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 76(5), pages 643-666, October.
    8. Arie Kapteyn & James P. Smith & Arthur Van Soest & Hana Vonkova, 2011. "Anchoring Vignettes and Response Consistency," Working Papers WR-840, RAND Corporation.
    9. Teresa Bago d’Uva & Maarten Lindeboom & Owen O’Donnell & Eddy van Doorslaer, 2011. "Slipping Anchor?: Testing the Vignettes Approach to Identification and Correction of Reporting Heterogeneity," Journal of Human Resources, University of Wisconsin Press, vol. 46(4), pages 875-906.
    10. Teresa Bago d'Uva & Eddy Van Doorslaer & Maarten Lindeboom & Owen O'Donnell, 2008. "Does reporting heterogeneity bias the measurement of health disparities?," Health Economics, John Wiley & Sons, Ltd., vol. 17(3), pages 351-375, March.
    11. Bertoni, Marco, 2015. "Hungry today, unhappy tomorrow? Childhood hunger and subjective wellbeing later in life," Journal of Health Economics, Elsevier, vol. 40(C), pages 40-53.
    12. Dolan, Paul & Kavetsos, Georgios & Tsuchiya, Aki, 2013. "Sick but satisfied: The impact of life and health satisfaction on choice between health scenarios," Journal of Health Economics, Elsevier, vol. 32(4), pages 708-714.
    13. Kristensen, Nicolai & Johansson, Edvard, 2008. "New evidence on cross-country differences in job satisfaction using anchoring vignettes," Labour Economics, Elsevier, vol. 15(1), pages 96-117, February.
    14. Franco Peracchi & Claudio Rossetti, 2012. "Heterogeneity in health responses and anchoring vignettes," Empirical Economics, Springer, vol. 42(2), pages 513-538, April.
    15. Jones, Andrew M. & Rice, Nigel & Robone, Silvana & Dias, Pedro Rosa, 2011. "Inequality and polarisation in health systems' responsiveness: A cross-country analysis," Journal of Health Economics, Elsevier, vol. 30(4), pages 616-625, July.
    16. Nancy J. Devlin & David Parkin & John Browne, 2010. "Patient‐reported outcome measures in the NHS: new methods for analysing and reporting EQ‐5D data," Health Economics, John Wiley & Sons, Ltd., vol. 19(8), pages 886-905, August.
    17. Richard Norman & Paula Cronin & Rosalie Viney, 2013. "A Pilot Discrete Choice Experiment to Explore Preferences for EQ-5D-5L Health States," Applied Health Economics and Health Policy, Springer, vol. 11(3), pages 287-298, June.
    18. Teresa Bago d'Uva & Maarten Lindeboom & Owen O'Donnell & Eddy van Doorslaer, 2011. "Education‐related inequity in healthcare with heterogeneous reporting of health," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 174(3), pages 639-664, July.
    19. Arie Kapteyn & James P. Smith & Arthur Van Soest & Hana Vonkova, 2011. "Anchoring Vignettes and Response Consistency," Working Papers 840, RAND Corporation.
    20. Jennifer Beam Dowd & Megan Todd, 2011. "Does Self-reported Health Bias the Measurement of Health Inequalities in U.S. Adults? Evidence Using Anchoring Vignettes From the Health and Retirement Study," The Journals of Gerontology: Series B, The Gerontological Society of America, vol. 66(4), pages 478-489.
    21. David K. Whynes & Nikola Sprigg & James Selby & Eivind Berge & Philip M. Bath, 2013. "Testing for Differential Item Functioning within the EQ-5D," Medical Decision Making, , vol. 33(2), pages 252-260, February.
    22. Hanna Grol-Prokopczyk & Emese Verdes-Tennant & Mary McEniry & Márton Ispány, 2015. "Promises and Pitfalls of Anchoring Vignettes in Health Survey Research," Demography, Springer;Population Association of America (PAA), vol. 52(5), pages 1703-1728, October.
    23. Teresa Molina, 2016. "Reporting Heterogeneity and Health Disparities Across Gender and Education Levels: Evidence From Four Countries," Demography, Springer;Population Association of America (PAA), vol. 53(2), pages 295-323, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ricardo Pagan & Miguel Ángel Malo, 2021. "Performance Appraisal and Job Satisfaction for Workers Without and With Disabilities by Gender," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 153(3), pages 1011-1039, February.
    2. Harris, Mark N. & Knott, Rachel J. & Lorgelly, Paula K. & Rice, Nigel, 2020. "Using externally collected vignettes to account for reporting heterogeneity in survey self-assessment," Economics Letters, Elsevier, vol. 194(C).
    3. Hana Vonkova, 2019. "Life Satisfaction among Different Groups of Children: Self-Reports, Differential Scale Usage and Anchoring Vignettes," Child Indicators Research, Springer;The International Society of Child Indicators (ISCI), vol. 12(6), pages 2111-2136, December.
    4. Greene, William & Harris, Mark N. & Knott, Rachel & Rice, Nigel, 2023. "Reporting heterogeneity in modeling self-assessed survey outcomes," Economic Modelling, Elsevier, vol. 124(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Knott, R. & Lorgelly, P. & Black, N. & Hollingsworth, B., 2016. "Differential item functioning in the EQ-5D: An exploratory analysis using anchoring vignettes," Health, Econometrics and Data Group (HEDG) Working Papers 16/14, HEDG, c/o Department of Economics, University of York.
    2. William H. Greene & Mark N. Harris & Rachel J. Knott & Nigel Rice, 2021. "Specification and testing of hierarchical ordered response models with anchoring vignettes," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(1), pages 31-64, January.
    3. Bertoni, Marco, 2015. "Hungry today, unhappy tomorrow? Childhood hunger and subjective wellbeing later in life," Journal of Health Economics, Elsevier, vol. 40(C), pages 40-53.
    4. Laura Rossouw & Teresa Bago d’Uva & Eddy Doorslaer, 2018. "Poor Health Reporting? Using Anchoring Vignettes to Uncover Health Disparities by Wealth and Race," Demography, Springer;Population Association of America (PAA), vol. 55(5), pages 1935-1956, October.
    5. Laura Rossouw & Teresa Bago d'Uva & Eddy van Doorslaer, 2017. "Poor Health Reporting? Using Vignettes to Recover the Health Gradient by Wealth," Tinbergen Institute Discussion Papers 17-031/V, Tinbergen Institute.
    6. Andrew M. Jones; Nigel Rice, Silvana Robone; & Nigel Rice; & Silvana Robone:, 2012. "A comparison of parametric and non-parametric adjustments using vignettes for self-reported data," Health, Econometrics and Data Group (HEDG) Working Papers 12/10, HEDG, c/o Department of Economics, University of York.
    7. Kapteyn, Arie & Wah, Saw Htay, 2016. "Challenges to small and medium-size businesses in Myanmar: What are they and how do we know?," Journal of Asian Economics, Elsevier, vol. 47(C), pages 1-22.
    8. Fontaine, Xavier & Haywood, Luke, 2018. "On the comparison of group inequalities using subjective data," Economics Letters, Elsevier, vol. 163(C), pages 17-21.
    9. Rachel J. Knott & Nicole Black & Bruce Hollingsworth & Paula K. Lorgelly, 2017. "Response‐Scale Heterogeneity in the EQ‐5D," Health Economics, John Wiley & Sons, Ltd., vol. 26(3), pages 387-394, March.
    10. Kaiser, Caspar, 2022. "Using memories to assess the intrapersonal comparability of wellbeing reports," Journal of Economic Behavior & Organization, Elsevier, vol. 193(C), pages 410-442.
    11. Silvana Robone, 2017. "Comment on ‘Response –scale heterogeneity in the EQ‐5D: Can we use vignettes to address response‐scale heterogeneity in the EQ‐5D? Not if but how’," Health Economics, John Wiley & Sons, Ltd., vol. 26(3), pages 395-397, March.
    12. Molina, Teresa, 2017. "Adjusting for heterogeneous response thresholds in cross-country comparisons of self-reported health," The Journal of the Economics of Ageing, Elsevier, vol. 10(C), pages 1-20.
    13. Montgomery, Mallory, 2022. "Reversing the gender gap in happiness," Journal of Economic Behavior & Organization, Elsevier, vol. 196(C), pages 65-78.
    14. Eric Bonsang & Arthur Soest, 2012. "Satisfaction with Job and Income Among Older Individuals Across European Countries," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 105(2), pages 227-254, January.
    15. Rachel J. Knott & Nicole Black & Bruce Hollingsworth & Paula K. Lorgelly, 2017. "Response to comment by robone: Practical advice for the implementation of anchoring vignettes," Health Economics, John Wiley & Sons, Ltd., vol. 26(3), pages 398-400, March.
    16. Franco Peracchi & Claudio Rossetti, 2013. "The heterogeneous thresholds ordered response model: identification and inference," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 176(3), pages 703-722, June.
    17. Raskina, Yulia & Podkorytova, Olga & Kuchakov, Ruslan, 2022. "Health determinants and the reporting heterogeneity bias in Russia: Anchoring vignettes approach," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 66, pages 118-143.
    18. Zhang, Hao & Bago d’Uva, Teresa & van Doorslaer, Eddy, 2015. "The gender health gap in China: A decomposition analysis," Economics & Human Biology, Elsevier, vol. 18(C), pages 13-26.
    19. Daniel J. Benjamin & Kristen Cooper & Ori Heffetz & Miles S. Kimball, 2023. "From Happiness Data to Economic Conclusions," NBER Working Papers 31727, National Bureau of Economic Research, Inc.
    20. Hendrik Jürges & Joachim Winter, 2013. "Are Anchoring Vignettes Ratings Sensitive To Vignette Age And Sex?," Health Economics, John Wiley & Sons, Ltd., vol. 22(1), pages 1-13, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:socmed:v:190:y:2017:i:c:p:247-255. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/315/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.