IDEAS home Printed from https://ideas.repec.org/a/sae/medema/v40y2020i3p314-326.html
   My bibliography  Save this article

Calculating the Expected Value of Sample Information in Practice: Considerations from 3 Case Studies

Author

Listed:
  • Anna Heath

    (The Hospital for Sick Children, Toronto, ON, Canada
    University of Toronto, Toronto, ON, Canada
    University College London, London, UK)

  • Natalia Kunst

    (Department of Health Management and Health Economics, Institute of Health and Society, Faculty of Medicine, University of Oslo, Oslo, Norway
    Cancer Outcomes, Public Policy and Effectiveness Research (COPPER) Center, Yale University School of Medicine and Yale Cancer Center, New Haven, CT, USA
    Department of Epidemiology and Biostatistics, Amsterdam UMC, Amsterdam, the Netherlands
    LINK Medical Research, Oslo, Norway)

  • Christopher Jackson

    (MRC Biostatistics Unit, University of Cambridge, Cambridge, UK)

  • Mark Strong

    (School of Health and Related Research, University of Sheffield, Sheffield, UK)

  • Fernando Alarid-Escudero

    (Center for Research and Teaching in Economics (CIDE))

  • Jeremy D. Goldhaber-Fiebert

    (Stanford Health Policy, Centers for Health Policy and Primary Care and Outcomes Research, Stanford University, Stanford, CA, USA)

  • Gianluca Baio

    (University College London, London, UK)

  • Nicolas A. Menzies

    (Harvard T. H. Chan School of Public Health, Boston, MA, USA)

  • Hawre Jalal

    (University of Pittsburgh, Pittsburgh, PA, USA)

Abstract

Background. Investing efficiently in future research to improve policy decisions is an important goal. Expected value of sample information (EVSI) can be used to select the specific design and sample size of a proposed study by assessing the benefit of a range of different studies. Estimating EVSI with the standard nested Monte Carlo algorithm has a notoriously high computational burden, especially when using a complex decision model or when optimizing over study sample sizes and designs. Recently, several more efficient EVSI approximation methods have been developed. However, these approximation methods have not been compared, and therefore their comparative performance across different examples has not been explored. Methods. We compared 4 EVSI methods using 3 previously published health economic models. The examples were chosen to represent a range of real-world contexts, including situations with multiple study outcomes, missing data, and data from an observational rather than a randomized study. The computational speed and accuracy of each method were compared. Results. In each example, the approximation methods took minutes or hours to achieve reasonably accurate EVSI estimates, whereas the traditional Monte Carlo method took weeks. Specific methods are particularly suited to problems where we wish to compare multiple proposed sample sizes, when the proposed sample size is large, or when the health economic model is computationally expensive. Conclusions. As all the evaluated methods gave estimates similar to those given by traditional Monte Carlo, we suggest that EVSI can now be efficiently computed with confidence in realistic examples. No systematically superior EVSI computation method exists as the properties of the different methods depend on the underlying health economic model, data generation process, and user expertise.

Suggested Citation

  • Anna Heath & Natalia Kunst & Christopher Jackson & Mark Strong & Fernando Alarid-Escudero & Jeremy D. Goldhaber-Fiebert & Gianluca Baio & Nicolas A. Menzies & Hawre Jalal, 2020. "Calculating the Expected Value of Sample Information in Practice: Considerations from 3 Case Studies," Medical Decision Making, , vol. 40(3), pages 314-326, April.
  • Handle: RePEc:sae:medema:v:40:y:2020:i:3:p:314-326
    DOI: 10.1177/0272989X20912402
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0272989X20912402
    Download Restriction: no

    File URL: https://libkey.io/10.1177/0272989X20912402?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Alan Brennan & Samer A. Kharroubi, 2007. "Expected value of sample information for Weibull survival data," Health Economics, John Wiley & Sons, Ltd., vol. 16(11), pages 1205-1225, November.
    2. Alan Brennan & Samer A. Kharroubi, 2007. "Expected value of sample information for Weibull survival data," Health Economics, John Wiley & Sons, Ltd., vol. 16(11), pages 1205-1225.
    3. Aaron A. Stinnett & John Mullahy, 1998. "Net Health Benefits: A New Framework for the Analysis of Uncertainty in Cost-Effectiveness Analysis," NBER Technical Working Papers 0227, National Bureau of Economic Research, Inc.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Mathyn Vervaart & Mark Strong & Karl P. Claxton & Nicky J. Welton & Torbjørn Wisløff & Eline Aas, 2022. "An Efficient Method for Computing Expected Value of Sample Information for Survival Data from an Ongoing Trial," Medical Decision Making, , vol. 42(5), pages 612-625, July.
    2. Anna Heath, 2022. "Calculating Expected Value of Sample Information Adjusting for Imperfect Implementation," Medical Decision Making, , vol. 42(5), pages 626-636, July.
    3. Nichola R. Naylor & Jack Williams & Nathan Green & Felicity Lamrock & Andrew Briggs, 2023. "Extensions of Health Economic Evaluations in R for Microsoft Excel Users: A Tutorial for Incorporating Heterogeneity and Conducting Value of Information Analyses," PharmacoEconomics, Springer, vol. 41(1), pages 21-32, January.
    4. Haitham Tuffaha & Claire Rothery & Natalia Kunst & Chris Jackson & Mark Strong & Stephen Birch, 2021. "A Review of Web-Based Tools for Value-of-Information Analysis," Applied Health Economics and Health Policy, Springer, vol. 19(5), pages 645-651, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Claire McKenna & Karl Claxton, 2011. "Addressing Adoption and Research Design Decisions Simultaneously," Medical Decision Making, , vol. 31(6), pages 853-865, November.
    2. Oakley, Jeremy E. & Brennan, Alan & Tappenden, Paul & Chilcott, Jim, 2010. "Simulation sample sizes for Monte Carlo partial EVPI calculations," Journal of Health Economics, Elsevier, vol. 29(3), pages 468-477, May.
    3. Stefano Conti & Karl Claxton, 2008. "Dimensions of design space: a decision-theoretic approach to optimal research design," Working Papers 038cherp, Centre for Health Economics, University of York.
    4. Nicolas A. Menzies, 2016. "An Efficient Estimator for the Expected Value of Sample Information," Medical Decision Making, , vol. 36(3), pages 308-320, April.
    5. Mark Strong & Jeremy E. Oakley & Alan Brennan & Penny Breeze, 2015. "Estimating the Expected Value of Sample Information Using the Probabilistic Sensitivity Analysis Sample," Medical Decision Making, , vol. 35(5), pages 570-583, July.
    6. Hawre Jalal & Jeremy D. Goldhaber-Fiebert & Karen M. Kuntz, 2015. "Computing Expected Value of Partial Sample Information from Probabilistic Sensitivity Analysis Using Linear Regression Metamodeling," Medical Decision Making, , vol. 35(5), pages 584-595, July.
    7. Anna Heath & Ioanna Manolopoulou & Gianluca Baio, 2018. "Efficient Monte Carlo Estimation of the Expected Value of Sample Information Using Moment Matching," Medical Decision Making, , vol. 38(2), pages 163-173, February.
    8. Anna Heath & Ioanna Manolopoulou & Gianluca Baio, 2019. "Estimating the Expected Value of Sample Information across Different Sample Sizes Using Moment Matching and Nonlinear Regression," Medical Decision Making, , vol. 39(4), pages 347-359, May.
    9. Andrew Willan & Simon Eckermann, 2012. "Value of Information and Pricing New Healthcare Interventions," PharmacoEconomics, Springer, vol. 30(6), pages 447-459, June.
    10. Hawre Jalal & Fernando Alarid-Escudero, 2018. "A Gaussian Approximation Approach for Value of Information Analysis," Medical Decision Making, , vol. 38(2), pages 174-188, February.
    11. Hendrik Koffijberg & Claire Rothery & Kalipso Chalkidou & Janneke Grutters, 2018. "Value of Information Choices that Influence Estimates: A Systematic Review of Prevailing Considerations," Medical Decision Making, , vol. 38(7), pages 888-900, October.
    12. Manuel Antonio Espinoza & Andrea Manca & Karl Claxton & Mark Sculpher, 2018. "Social value and individual choice: The value of a choice‐based decision‐making process in a collectively funded health system," Health Economics, John Wiley & Sons, Ltd., vol. 27(2), pages 28-40, February.
    13. A. E. Ades & Karl Claxton & Mark Sculpher, 2006. "Evidence synthesis, parameter correlation and probabilistic sensitivity analysis," Health Economics, John Wiley & Sons, Ltd., vol. 15(4), pages 373-381, April.
    14. Basu, Anirban & Jena, Anupam B. & Philipson, Tomas J., 2011. "The impact of comparative effectiveness research on health and health care spending," Journal of Health Economics, Elsevier, vol. 30(4), pages 695-706, July.
    15. Karl Claxton & Elisabeth Fenwick & Mark J. Sculpher, 2012. "Decision-making with Uncertainty: The Value of Information," Chapters, in: Andrew M. Jones (ed.), The Elgar Companion to Health Economics, Second Edition, chapter 51, Edward Elgar Publishing.
    16. Simon Eckermann & Tim Coelli, 2008. "Including quality attributes in a model of health care efficiency: A net benefit approach," CEPA Working Papers Series WP032008, School of Economics, University of Queensland, Australia.
    17. Frank G. Sandmann & Julie V. Robotham & Sarah R. Deeny & W. John Edmunds & Mark Jit, 2018. "Estimating the opportunity costs of bed‐days," Health Economics, John Wiley & Sons, Ltd., vol. 27(3), pages 592-605, March.
    18. Clarke, Philip M. & Hayes, Alison J., 2009. "Measuring achievement: Changes in risk factors for cardiovascular disease in Australia," Social Science & Medicine, Elsevier, vol. 68(3), pages 552-561, February.
    19. Niklas Zethraeus & Magnus Johannesson & Bengt Jönsson & Mickael Löthgren & Magnus Tambour, 2003. "Advantages of Using the Net-Benefit Approach for Analysing Uncertainty in Economic Evaluation Studies," PharmacoEconomics, Springer, vol. 21(1), pages 39-48, January.
    20. Moreno, Elías & Girón, F.J. & Vázquez-Polo, F.J. & Negrín, M.A., 2012. "Optimal healthcare decisions: The importance of the covariates in cost–effectiveness analysis," European Journal of Operational Research, Elsevier, vol. 218(2), pages 512-522.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:medema:v:40:y:2020:i:3:p:314-326. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.