IDEAS home Printed from https://ideas.repec.org/a/eee/jhecon/v29y2010i3p468-477.html
   My bibliography  Save this article

Simulation sample sizes for Monte Carlo partial EVPI calculations

Author

Listed:
  • Oakley, Jeremy E.
  • Brennan, Alan
  • Tappenden, Paul
  • Chilcott, Jim

Abstract

Partial expected value of perfect information (EVPI) quantifies the value of removing uncertainty about unknown parameters in a decision model. EVPIs can be computed via Monte Carlo methods. An outer loop samples values of the parameters of interest, and an inner loop samples the remaining parameters from their conditional distribution. This nested Monte Carlo approach can result in biased estimates if small numbers of inner samples are used and can require a large number of model runs for accurate partial EVPI estimates. We present a simple algorithm to estimate the EVPI bias and confidence interval width for a specified number of inner and outer samples. The algorithm uses a relatively small number of model runs (we suggest approximately 600), is quick to compute, and can help determine how many outer and inner iterations are needed for a desired level of accuracy. We test our algorithm using three case studies.

Suggested Citation

  • Oakley, Jeremy E. & Brennan, Alan & Tappenden, Paul & Chilcott, Jim, 2010. "Simulation sample sizes for Monte Carlo partial EVPI calculations," Journal of Health Economics, Elsevier, vol. 29(3), pages 468-477, May.
    • Handle: RePEc:eee:jhecon:v:29:y:2010:i:3:p:468-477
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167-6296(10)00047-0
    Download Restriction: Full text for ScienceDirect subscribers only
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. M. D. Stevenson & J. Oakley & J. B. Chilcott, 2004. "Gaussian Process Modeling in Conjunction with Individual Patient Simulation Modeling: A Case Study Describing the Calculation of Cost-Effectiveness Ratios for the Treatment of Established Osteoporosis," Medical Decision Making, , vol. 24(1), pages 89-100, January.
    2. Alan Brennan & Samer A. Kharroubi, 2007. "Expected value of sample information for Weibull survival data," Health Economics, John Wiley & Sons, Ltd., vol. 16(11), pages 1205-1225, November.
    3. Elisabeth Fenwick & Karl Claxton & Mark Sculpher, 2005. "The value of implementation and the value of information: combined and uneven development," Working Papers 005cherp, Centre for Health Economics, University of York.
    4. Jeremy E. Oakley & Anthony O'Hagan, 2004. "Probabilistic sensitivity analysis of complex models: a Bayesian approach," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 66(3), pages 751-769, August.
    5. Alan Brennan & Samer A. Kharroubi, 2007. "Expected value of sample information for Weibull survival data," Health Economics, John Wiley & Sons, Ltd., vol. 16(11), pages 1205-1225.
    6. Meltzer, David, 2001. "Addressing uncertainty in medical cost-effectiveness analysis: Implications of expected utility maximization for methods to perform sensitivity analysis and the use of cost-effectiveness analysis to s," Journal of Health Economics, Elsevier, vol. 20(1), pages 109-129, January.
    7. A. E. Ades & G. Lu & K. Claxton, 2004. "Expected Value of Sample Information Calculations in Medical Decision Modeling," Medical Decision Making, , vol. 24(2), pages 207-227, March.
    8. Bas Groot Koerkamp & M. G. Myriam Hunink & Theo Stijnen & Milton C. Weinstein, 2006. "Identifying key parameters in cost‐effectiveness analysis using value of information: a comparison of methods," Health Economics, John Wiley & Sons, Ltd., vol. 15(4), pages 383-392, April.
    9. Anthony O'Hagan & Matt Stevenson & Jason Madan, 2007. "Monte Carlo probabilistic sensitivity analysis for patient level simulation models: efficient estimation of mean and variance using ANOVA," Health Economics, John Wiley & Sons, Ltd., vol. 16(10), pages 1009-1023, October.
    10. Fumie Yokota & Kimberly M. Thompson, 2004. "Value of Information Literature Analysis: A Review of Applications in Health Risk Management," Medical Decision Making, , vol. 24(3), pages 287-298, June.
    11. Alan Brennan & Stephen E. Chick & Ruth Davies, 2006. "A taxonomy of model structures for economic evaluation of health technologies," Health Economics, John Wiley & Sons, Ltd., vol. 15(12), pages 1295-1310, December.
    12. Anthony O'Hagan & Matt Stevenson & Jason Madan, 2007. "Monte Carlo probabilistic sensitivity analysis for patient level simulation models: efficient estimation of mean and variance using ANOVA," Health Economics, John Wiley & Sons, Ltd., vol. 16(10), pages 1009-1023.
    13. Karl Claxton & John Posnett, "undated". "An Economic Approach to Clinical Trial Design and Research Priority Setting," Discussion Papers 96/19, Department of Economics, University of York.
    14. James C. Felli & Gordon B. Hazen, 1998. "Sensitivity Analysis and the Expected Value of Perfect Information," Medical Decision Making, , vol. 18(1), pages 95-109, January.
    15. Brennan, Alan & Kharroubi, Samer A., 2007. "Efficient computation of partial expected value of sample information using Bayesian approximation," Journal of Health Economics, Elsevier, vol. 26(1), pages 122-148, January.
    16. Alan Brennan & Samer Kharroubi & Anthony O'Hagan & Jim Chilcott, 2007. "Calculating Partial Expected Value of Perfect Information via Monte Carlo Sampling Algorithms," Medical Decision Making, , vol. 27(4), pages 448-470, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Gao, Lei & Bryan, Brett A., 2016. "Incorporating deep uncertainty into the elementary effects method for robust global sensitivity analysis," Ecological Modelling, Elsevier, vol. 321(C), pages 1-9.
    2. Malings, Carl & Pozzi, Matteo, 2016. "Value of information for spatially distributed systems: Application to sensor placement," Reliability Engineering and System Safety, Elsevier, vol. 154(C), pages 219-233.
    3. Laura McCullagh & Cathal Walsh & Michael Barry, 2012. "Value-of-Information Analysis to Reduce Decision Uncertainty Associated with the Choice of Thromboprophylaxis after Total Hip Replacement in the Irish Healthcare Setting," PharmacoEconomics, Springer, vol. 30(10), pages 941-959, October.
    4. Isaac Corro Ramos & Maureen P. M. H. Rutten-van Mölken & Maiwenn J. Al, 2013. "The Role of Value-of-Information Analysis in a Health Care Research Priority Setting," Medical Decision Making, , vol. 33(4), pages 472-489, May.
    5. Anna Heath & Ioanna Manolopoulou & Gianluca Baio, 2018. "Efficient Monte Carlo Estimation of the Expected Value of Sample Information Using Moment Matching," Medical Decision Making, , vol. 38(2), pages 163-173, February.
    6. Plischke, Elmar & Borgonovo, Emanuele & Smith, Curtis L., 2013. "Global sensitivity measures from given data," European Journal of Operational Research, Elsevier, vol. 226(3), pages 536-550.
    7. Gordon Hazen & Emanuele Borgonovo & Xuefei Lu, 2023. "Information Density in Decision Analysis," Decision Analysis, INFORMS, vol. 20(2), pages 89-108, June.
    8. Emanuele Borgonovo & Gordon B. Hazen & Elmar Plischke, 2016. "A Common Rationale for Global Sensitivity Measures and Their Estimation," Risk Analysis, John Wiley & Sons, vol. 36(10), pages 1871-1895, October.
    9. Haag, Fridolin & Chennu, Arjun, 2023. "Assessing whether decisions are more sensitive to preference or prediction uncertainty with a value of information approach," Omega, Elsevier, vol. 121(C).
    10. Nicky J. Welton & Jason J. Madan & Deborah M. Caldwell & Tim J. Peters & Anthony E. Ades, 2014. "Expected Value of Sample Information for Multi-Arm Cluster Randomized Trials with Binary Outcomes," Medical Decision Making, , vol. 34(3), pages 352-365, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Stefano Conti & Karl Claxton, 2008. "Dimensions of design space: a decision-theoretic approach to optimal research design," Working Papers 038cherp, Centre for Health Economics, University of York.
    2. Rachael L. Fleurence, 2007. "Setting priorities for research: a practical application of 'payback' and expected value of information," Health Economics, John Wiley & Sons, Ltd., vol. 16(12), pages 1345-1357.
    3. Alan Brennan & Samer A. Kharroubi, 2007. "Expected value of sample information for Weibull survival data," Health Economics, John Wiley & Sons, Ltd., vol. 16(11), pages 1205-1225, November.
    4. Brennan, Alan & Kharroubi, Samer A., 2007. "Efficient computation of partial expected value of sample information using Bayesian approximation," Journal of Health Economics, Elsevier, vol. 26(1), pages 122-148, January.
    5. Alan Brennan & Samer A. Kharroubi, 2007. "Expected value of sample information for Weibull survival data," Health Economics, John Wiley & Sons, Ltd., vol. 16(11), pages 1205-1225.
    6. Andrew Willan & Simon Eckermann, 2012. "Value of Information and Pricing New Healthcare Interventions," PharmacoEconomics, Springer, vol. 30(6), pages 447-459, June.
    7. Marta Soares & Luísa Canto e Castro, 2012. "Continuous Time Simulation and Discretized Models for Cost-Effectiveness Analysis," PharmacoEconomics, Springer, vol. 30(12), pages 1101-1117, December.
    8. Marta O Soares & L Canto e Castro, 2010. "Simulation or cohort models? Continuous time simulation and discretized Markov models to estimate cost-effectiveness," Working Papers 056cherp, Centre for Health Economics, University of York.
    9. Marta O. Soares & Luísa Canto e Castro, 2012. "Continuous Time Simulation and Discretized Models for Cost-Effectiveness Analysis," PharmacoEconomics, Springer, vol. 30(12), pages 1101-1117, December.
    10. Jonathan Karnon & James Stahl & Alan Brennan & J. Jaime Caro & Javier Mar & Jörgen Möller, 2012. "Modeling Using Discrete Event Simulation," Medical Decision Making, , vol. 32(5), pages 701-711, September.
    11. Samer A. Kharroubi & Alan Brennan & Mark Strong, 2011. "Estimating Expected Value of Sample Information for Incomplete Data Models Using Bayesian Approximation," Medical Decision Making, , vol. 31(6), pages 839-852, November.
    12. V. J. Roelofs & M. C. Kennedy, 2011. "Sensitivity Analysis and Estimation of Extreme Tail Behavior in Two‐Dimensional Monte Carlo Simulation," Risk Analysis, John Wiley & Sons, vol. 31(10), pages 1597-1609, October.
    13. Mark Strong & Jeremy E. Oakley & Alan Brennan & Penny Breeze, 2015. "Estimating the Expected Value of Sample Information Using the Probabilistic Sensitivity Analysis Sample," Medical Decision Making, , vol. 35(5), pages 570-583, July.
    14. Fleurence, Rachael L. & Torgerson, David J., 2004. "Setting priorities for research," Health Policy, Elsevier, vol. 69(1), pages 1-10, July.
    15. Bas Groot Koerkamp & M. G. Myriam Hunink & Theo Stijnen & Milton C. Weinstein, 2006. "Identifying key parameters in cost‐effectiveness analysis using value of information: a comparison of methods," Health Economics, John Wiley & Sons, Ltd., vol. 15(4), pages 383-392, April.
    16. Hawre Jalal & Jeremy D. Goldhaber-Fiebert & Karen M. Kuntz, 2015. "Computing Expected Value of Partial Sample Information from Probabilistic Sensitivity Analysis Using Linear Regression Metamodeling," Medical Decision Making, , vol. 35(5), pages 584-595, July.
    17. Anthony O'Hagan & Matt Stevenson & Jason Madan, 2007. "Monte Carlo probabilistic sensitivity analysis for patient level simulation models: efficient estimation of mean and variance using ANOVA," Health Economics, John Wiley & Sons, Ltd., vol. 16(10), pages 1009-1023, October.
    18. Rachael L. Fleurence, 2007. "Setting priorities for research: a practical application of ‘payback’ and expected value of information," Health Economics, John Wiley & Sons, Ltd., vol. 16(12), pages 1345-1357, December.
    19. Anna Heath & Ioanna Manolopoulou & Gianluca Baio, 2018. "Efficient Monte Carlo Estimation of the Expected Value of Sample Information Using Moment Matching," Medical Decision Making, , vol. 38(2), pages 163-173, February.
    20. Joke Bilcke & Philippe Beutels & Marc Brisson & Mark Jit, 2011. "Accounting for Methodological, Structural, and Parameter Uncertainty in Decision-Analytic Models," Medical Decision Making, , vol. 31(4), pages 675-692, July.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jhecon:v:29:y:2010:i:3:p:468-477. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/inca/505560 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.