IDEAS home Printed from https://ideas.repec.org/a/spr/eujhec/v22y2021i1d10.1007_s10198-020-01224-6.html
   My bibliography  Save this article

Increasing respondent engagement in composite time trade-off tasks by imposing three minimum trade-offs to improve data quality

Author

Listed:
  • Ruixuan Jiang

    (Merck & Co.)

  • Thomas Kohlmann

    (Medical University Greifswald)

  • Todd A. Lee

    (University of Illinois at Chicago College of Pharmacy)

  • Axel Mühlbacher

    (Hochschule Neubrandenburg)

  • James Shaw

    (Bristol-Myers Squibb)

  • Surrey Walton

    (University of Illinois at Chicago College of Pharmacy)

  • A. Simon Pickard

    (University of Illinois at Chicago College of Pharmacy)

Abstract

Background Web-based surveys are increasingly utilized for health valuation studies but may be more prone to lack of engagement and, therefore, poor data validity. The objective of this study was to evaluate the effect of imposed engagement (i.e., at least three trade-offs) in the composite time trade-off (cTTO) task. Methods The EQ-5D-5L valuation study protocol and study design were adapted for online, unsupervised completion in two arms: base case and engagement. Validity of preferences was assessed using the prevalence of inconsistent valuations and expected patterns of TTO values. Respondent task engagement was measured using time per task. Value sets were generated using linear regression with a random intercept (RILR). Results The base case (n = 501) and engagement arms (n = 504) clustered at different TTO values: [base case] 0, 1; [engagement] -0.5, 0.45, 0.6. Mean TTO values were lower for the engagement arm. Engagement respondents did not spend more time per TTO task: [base case] 63.3 s (SD 77.9 s); [engagement] 64.7 s (SD 73.3 s); p = 0.36. No significant difference was found between arms for prevalence of respondents with at least one inconsistent TTO value: [base case] 61.1%; [engagement] 63.5%; p = 0.43. Both value sets had significant intercepts far from 1: [base case] 0.846; [engagement] 0.783. The relative importance of the EQ-5D dimensions also differed between arms. Conclusions Both online arms had poor quality data. A minimum trade-off threshold did not improve engagement nor face validity of the data, indicating that modifications to the number of iterations are insufficient alone to improve data quality/validity of online TTO studies.

Suggested Citation

  • Ruixuan Jiang & Thomas Kohlmann & Todd A. Lee & Axel Mühlbacher & James Shaw & Surrey Walton & A. Simon Pickard, 2021. "Increasing respondent engagement in composite time trade-off tasks by imposing three minimum trade-offs to improve data quality," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 22(1), pages 17-33, February.
    • Handle: RePEc:spr:eujhec:v:22:y:2021:i:1:d:10.1007_s10198-020-01224-6
    DOI: 10.1007/s10198-020-01224-6
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10198-020-01224-6
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10198-020-01224-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Nick Bansback & Aki Tsuchiya & John Brazier & Aslam Anis, 2012. "Canadian Valuation of EQ-5D Health States: Preliminary Value Set and Considerations for Future Valuation Studies," PLOS ONE, Public Library of Science, vol. 7(2), pages 1-11, February.
    2. Nancy Devlin & Ken Buckingham & Koonal Shah & Aki Tsuchiya & Carl Tilling & Grahame Wilkinson & Ben van Hout, 2013. "A Comparison Of Alternative Variants Of The Lead And Lag Time Tto," Health Economics, John Wiley & Sons, Ltd., vol. 22(5), pages 517-532, May.
    3. Leslie A. Lenert & Ann Sturley & Marcia Rupnow, 2003. "Toward Improved Methods for Measurement of Utility: Automated Repair of Errors in Elicitations," Medical Decision Making, , vol. 23(1), pages 67-75, January.
    4. Feng Xie & Eleanor Pullenayegum & A. Simon Pickard & Juan Manuel Ramos Goñi & Min‐woo Jo & Ataru Igarashi, 2017. "Transforming Latent Utilities to Health Utilities: East Does Not Meet West," Health Economics, John Wiley & Sons, Ltd., vol. 26(12), pages 1524-1533, December.
    5. Nan Luo & Minghui Li & Elly Stolk & Nancy Devlin, 2013. "The effects of lead time and visual aids in TTO valuation: a study of the EQ-VT framework," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 14(1), pages 15-24, July.
    6. Leslie A. Lenert & Daniel J. Cher & Mary K. Goldstein & Merlynn R. Bergen & Alan Garber, 1998. "The Effect of Search Procedures on Utility Elicitations," Medical Decision Making, , vol. 18(1), pages 76-83, January.
    7. Yvette Edelaar-Peeters & Anne M. Stiggelbout & Wilbert B. Van Den Hout, 2014. "Qualitative and Quantitative Analysis of Interviewer Help Answering the Time Tradeoff," Medical Decision Making, , vol. 34(5), pages 655-665, July.
    8. Mark Oppe & Kim Rand-Hendriksen & Koonal Shah & Juan M. Ramos‐Goñi & Nan Luo, 2016. "EuroQol Protocols for Time Trade-Off Valuation of Health Outcomes," PharmacoEconomics, Springer, vol. 34(10), pages 993-1004, October.
    9. Leonard J. Paas & Meike Morren, 2018. "PLease do not answer if you are reading this: respondent attention in online panels," Marketing Letters, Springer, vol. 29(1), pages 13-21, March.
    10. Matthijs Versteegh & Arthur Attema & Mark Oppe & Nancy Devlin & Elly Stolk, 2013. "Time to tweak the TTO: results from a comparison of alternative specifications of the TTO," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 14(1), pages 43-51, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Blog mentions

    As found by EconAcademics.org, the blog aggregator for Economics research:
    1. Rita Faria’s journal round-up for 8th February 2021
      by Rita Faria in The Academic Health Economists' Blog on 2021-02-08 12:00:01

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jen-Yu Amy Chang & Chien-Ning Hsu & Juan Manuel Ramos-Goñi & Nan Luo & Hsiang-Wen Lin & Fang-Ju Lin, 2024. "Beyond 10-year lead-times in EQ-5D-5L: leveraging alternative lead-times in willingness-to-accept questions to capture preferences for worse-than-dead states and their implication," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 25(6), pages 1041-1055, August.
    2. Arthur Attema & Yvette Edelaar-Peeters & Matthijs Versteegh & Elly Stolk, 2013. "Time trade-off: one methodology, different methods," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 14(1), pages 53-64, July.
    3. Koonal K. Shah & Juan Manuel Ramos-Goñi & Simone Kreimeier & Nancy J. Devlin, 2020. "An exploration of methods for obtaining 0 = dead anchors for latent scale EQ-5D-Y values," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 21(7), pages 1091-1103, September.
    4. Stefan A. Lipman & Liying Zhang & Koonal K. Shah & Arthur E. Attema, 2023. "Time and lexicographic preferences in the valuation of EQ-5D-Y with time trade-off methodology," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 24(2), pages 293-305, March.
    5. Finch, Aureliano Paolo & Meregaglia, Michela & Ciani, Oriana & Roudijk, Bram & Jommi, Claudio, 2022. "An EQ-5D-5L value set for Italy using videoconferencing interviews and feasibility of a new mode of administration," Social Science & Medicine, Elsevier, vol. 292(C).
    6. Versteegh, MM & Attema, AE & Oppe, M & Devlin, NJ & Stolk, EA, 2012. "Time to tweak the TTO. But how?," MPRA Paper 37989, University Library of Munich, Germany.
    7. P. Wang & M. Li & G. Liu & J. Thumboo & N. Luo, 2015. "Do Chinese have similar health-state preferences? A comparison of mainland Chinese and Singaporean Chinese," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 16(8), pages 857-863, November.
    8. Matthijs Versteegh & Arthur Attema & Mark Oppe & Nancy Devlin & Elly Stolk, 2013. "Time to tweak the TTO: results from a comparison of alternative specifications of the TTO," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 14(1), pages 43-51, July.
    9. Hobbins, Anna P. & Barry, Luke & Kelleher, Dan & Shah, Koonal & Devlin, Nancy & Ramos Goni, Juan Manuel & O’Neill, Ciaran, 2020. "Do people with private health insurance attach a higher value to health than those without insurance? Results from an EQ-5D-5 L valuation study in Ireland," Health Policy, Elsevier, vol. 124(6), pages 639-646.
    10. Baier-Fuentes, Hugo & Andrade-Valbuena, Nelson A. & Huertas Gonzalez-Serrano, Maria & Gaviria-Marin, Magaly, 2023. "Bricolage as an effective tool for the survival of owner-managed SMEs during crises," Journal of Business Research, Elsevier, vol. 157(C).
    11. Juan Ramos-Goñi & Oliver Rivero-Arias & María Errea & Elly Stolk & Michael Herdman & Juan Cabasés, 2013. "Dealing with the health state ‘dead’ when using discrete choice experiments to obtain values for EQ-5D-5L heath states," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 14(1), pages 33-42, July.
    12. Touré, Moustapha & Poder, Thomas G., 2024. "Differences in health utilities between cancer patients and the general population: The case of Quebec using the SF-6Dv2," Social Science & Medicine, Elsevier, vol. 351(C).
    13. Koonal Shah & Andrew Lloyd & Mark Oppe & Nancy Devlin, 2013. "One-to-one versus group setting for conducting computer-assisted TTO studies: findings from pilot studies in England and the Netherlands," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 14(1), pages 65-73, July.
    14. Patricia Cubi-Molla & Koonal Shah & Kristina Burström, 2018. "Experience-Based Values: A Framework for Classifying Different Types of Experience in Health Valuation Research," The Patient: Patient-Centered Outcomes Research, Springer;International Academy of Health Preference Research, vol. 11(3), pages 253-270, June.
    15. Schwappach, David L.B. & Strasmann, Thomas J., 2006. ""Quick and dirty numbers"?: The reliability of a stated-preference technique for the measurement of preferences for resource allocation," Journal of Health Economics, Elsevier, vol. 25(3), pages 432-448, May.
    16. Alessandro Gandolfo, 2021. "The Drawbacks of the Digital Transition of Marketing Research: Implications for Decision Makers and the Industry," International Journal of Marketing Studies, Canadian Center of Science and Education, vol. 13(3), pages 1-9, December.
    17. Kostyk, Alena & Zhou, Wenkai & Hyman, Michael R., 2019. "Using surveytainment to counter declining survey data quality," Journal of Business Research, Elsevier, vol. 95(C), pages 211-219.
    18. Han Bleichrodt, 2002. "A new explanation for the difference between time trade‐off utilities and standard gamble utilities," Health Economics, John Wiley & Sons, Ltd., vol. 11(5), pages 447-456, July.
    19. Trude Arnesen & Mari Trommald, 2005. "Are QALYs based on time trade‐off comparable? – A systematic review of TTO methodologies," Health Economics, John Wiley & Sons, Ltd., vol. 14(1), pages 39-53, January.
    20. Hämäläinen, Raimo P. & Lahtinen, Tuomas J., 2016. "Path dependence in Operational Research—How the modeling process can influence the results," Operations Research Perspectives, Elsevier, vol. 3(C), pages 14-20.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:eujhec:v:22:y:2021:i:1:d:10.1007_s10198-020-01224-6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.