IDEAS home Printed from https://ideas.repec.org/a/sae/medema/v39y2019i6p693-703.html
   My bibliography  Save this article

Exclusion Criteria as Measurements I: Identifying Invalid Responses

Author

Listed:
  • Barry Dewitt

    (Department of Engineering & Public Policy, Carnegie Mellon University, Pittsburgh, PA, USA)

  • Baruch Fischhoff

    (Department of Engineering & Public Policy, Carnegie Mellon University, Pittsburgh, PA, USA
    The Institute for Politics and Strategy, Carnegie Mellon University, Pittsburgh, PA, USA)

  • Alexander L. Davis

    (Department of Engineering & Public Policy, Carnegie Mellon University, Pittsburgh, PA, USA)

  • Stephen B. Broomell

    (Department of Social and Decision Sciences, Carnegie Mellon University, Pittsburgh, PA, USA)

  • Mark S. Roberts

    (Division of General Internal Medicine, University of Pittsburgh, Pittsburgh, PA, USA
    Department of Health Policy and Management, Graduate School of Public Health, University of Pittsburgh, Pittsburgh, PA, USA)

  • Janel Hanmer

    (Division of General Internal Medicine, University of Pittsburgh, Pittsburgh, PA, USA)

Abstract

Background. In a systematic review, Engel et al. found large variation in the exclusion criteria used to remove responses held not to represent genuine preferences in health state valuation studies. We offer an empirical approach to characterizing the similarities and differences among such criteria. Setting. Our analyses use data from an online survey that elicited preferences for health states defined by domains from the Patient-Reported Outcomes Measurement Information System (PROMIS ® ), with a U.S. nationally representative sample ( N = 1164). Methods. We use multidimensional scaling to investigate how 10 commonly used exclusion criteria classify participants and their responses. Results. We find that the effects of exclusion criteria do not always match the reasons advanced for applying them. For example, excluding very high and very low values has been justified as removing aberrant responses. However, people who give very high and very low values prove to be systematically different in ways suggesting that such responses may reflect different processes. Conclusions. Exclusion criteria intended to remove low-quality responses from health state valuation studies may actually remove deliberate but unusual ones. A companion article examines the effects of the exclusion criteria on societal utility estimates.

Suggested Citation

  • Barry Dewitt & Baruch Fischhoff & Alexander L. Davis & Stephen B. Broomell & Mark S. Roberts & Janel Hanmer, 2019. "Exclusion Criteria as Measurements I: Identifying Invalid Responses," Medical Decision Making, , vol. 39(6), pages 693-703, August.
    • Handle: RePEc:sae:medema:v:39:y:2019:i:6:p:693-703
    DOI: 10.1177/0272989X19856617
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0272989X19856617
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0272989X19856617?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. J. Gower & P. Legendre, 1986. "Metric and Euclidean properties of dissimilarity coefficients," Journal of Classification, Springer;The Classification Society, vol. 3(1), pages 5-48, March.
    2. Julie Ratcliffe & John Brazier & Aki Tsuchiya & Tara Symonds & Martin Brown, 2009. "Using DCE and ranking data to estimate cardinal values for health states for deriving a preference‐based single index from the sexual quality of life questionnaire," Health Economics, John Wiley & Sons, Ltd., vol. 18(11), pages 1261-1276, November.
    3. Fischhoff, Baruch & Kadvany, John, 2011. "Risk: A Very Short Introduction," OUP Catalogue, Oxford University Press, number 9780199576203.
    4. Nancy J. Devlin & Paul Hansen & Paul Kind & Alan Williams, 2003. "Logical inconsistencies in survey respondents' health state valuations ‐ a methodological challenge for estimating social tariffs," Health Economics, John Wiley & Sons, Ltd., vol. 12(7), pages 529-544, July.
    5. Versteegh, M.M. & Brouwer, W.B.F., 2016. "Patient and general public preferences for health states: A call to reconsider current guidelines," Social Science & Medicine, Elsevier, vol. 165(C), pages 66-74.
    6. Jan Van den Broeck & Solveig Argeseanu Cunningham & Roger Eeckels & Kobus Herbst, 2005. "Data Cleaning: Detecting, Diagnosing, and Editing Data Abnormalities," PLOS Medicine, Public Library of Science, vol. 2(10), pages 1-1, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Barry Dewitt & Baruch Fischhoff & Alexander L. Davis & Stephen B. Broomell & Mark S. Roberts & Janel Hanmer, 2019. "Exclusion Criteria as Measurements II: Effects on Utility Functions," Medical Decision Making, , vol. 39(6), pages 704-716, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Bansback, Nick & Brazier, John & Tsuchiya, Aki & Anis, Aslam, 2010. "Using a discrete choice experiment to estimate societal health state utility values," MPRA Paper 29933, University Library of Munich, Germany.
    2. Mina Bahrampour & Joshua Byrnes & Richard Norman & Paul A. Scuffham & Martin Downes, 2020. "Discrete choice experiments to generate utility values for multi-attribute utility instruments: a systematic review of methods," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 21(7), pages 983-992, September.
    3. Moustapha Touré & Christian R. C. Kouakou & Thomas G. Poder, 2021. "Dimensions Used in Instruments for QALY Calculation: A Systematic Review," IJERPH, MDPI, vol. 18(9), pages 1-22, April.
    4. Bansback, Nick & Brazier, John & Tsuchiya, Aki & Anis, Aslam, 2012. "Using a discrete choice experiment to estimate health state utility values," Journal of Health Economics, Elsevier, vol. 31(1), pages 306-318.
    5. Guohuan Su & Adam Mertel & Sébastien Brosse & Justin M. Calabrese, 2023. "Species invasiveness and community invasibility of North American freshwater fish fauna revealed via trait-based analysis," Nature Communications, Nature, vol. 14(1), pages 1-12, December.
    6. Juan Ramos-Goñi & Oliver Rivero-Arias & María Errea & Elly Stolk & Michael Herdman & Juan Cabasés, 2013. "Dealing with the health state ‘dead’ when using discrete choice experiments to obtain values for EQ-5D-5L heath states," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 14(1), pages 33-42, July.
    7. Stevens, K, 2010. "Valuation of the Child Health Utility Index 9D (CHU9D)," MPRA Paper 29938, University Library of Munich, Germany.
    8. Balepur, Prashant Narayan, 1998. "Impacts of Computer-Mediated Communication on Travel and Communication Patterns: The Davis Community Network Study," Institute of Transportation Studies, Research Reports, Working Papers, Proceedings qt6cb1f85c, Institute of Transportation Studies, UC Berkeley.
    9. Douglas L. Steinley & M. J. Brusco, 2019. "Using an Iterative Reallocation Partitioning Algorithm to Verify Test Multidimensionality," Journal of Classification, Springer;The Classification Society, vol. 36(3), pages 397-413, October.
    10. Wallin, Annika & Wahlberg, Lena & Persson, Johannes & Dewitt, Barry, 2020. "“Science and proven experience”: How should the epistemology of medicine inform the regulation of healthcare?," Health Policy, Elsevier, vol. 124(8), pages 842-848.
    11. Anna Maria D’Arcangelis & Giulia Rotundo, 2016. "Complex Networks in Finance," Lecture Notes in Economics and Mathematical Systems, in: Pasquale Commendatore & Mariano Matilla-García & Luis M. Varela & Jose S. Cánovas (ed.), Complex Networks and Dynamics, pages 209-235, Springer.
    12. Carla Coltharp & Rene P Kessler & Jie Xiao, 2012. "Accurate Construction of Photoactivated Localization Microscopy (PALM) Images for Quantitative Measurements," PLOS ONE, Public Library of Science, vol. 7(12), pages 1-15, December.
    13. de Hond, Anne & Bakx, Pieter & Versteegh, Matthijs, 2019. "Can time heal all wounds? An empirical assessment of adaptation to functional limitations in an older population," Social Science & Medicine, Elsevier, vol. 222(C), pages 180-187.
    14. S. T. Buckland & Y. Yuan & E. Marcon, 2017. "Measuring temporal trends in biodiversity," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 101(4), pages 461-474, October.
    15. Matthijs Warrens, 2008. "On the Indeterminacy of Resemblance Measures for Binary (Presence/Absence) Data," Journal of Classification, Springer;The Classification Society, vol. 25(1), pages 125-136, June.
    16. Stefano Bonnini & Getnet Melak Assegie & Kamila Trzcinska, 2024. "Review about the Permutation Approach in Hypothesis Testing," Mathematics, MDPI, vol. 12(17), pages 1-29, August.
    17. Kivimäki, Ilkka & Shimbo, Masashi & Saerens, Marco, 2014. "Developments in the theory of randomized shortest paths with a comparison of graph node distances," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 393(C), pages 600-616.
    18. Stöckel, Jannis & van Exel, Job & Brouwer, Werner B.F., 2023. "Adaptation in life satisfaction and self-assessed health to disability - Evidence from the UK," Social Science & Medicine, Elsevier, vol. 328(C).
    19. Paul F M Krabbe, 2013. "A Generalized Measurement Model to Quantify Health: The Multi-Attribute Preference Response Model," PLOS ONE, Public Library of Science, vol. 8(11), pages 1-12, November.
    20. Alfonso Gutierrez-Lopez & Carlos Chávez & Carlos Díaz-Delgado, 2022. "Autocorrelation Ratio as a Measure of Inertia for the Classification of Extreme Events," Mathematics, MDPI, vol. 10(12), pages 1-15, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:medema:v:39:y:2019:i:6:p:693-703. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.