IDEAS home Printed from https://ideas.repec.org/a/sae/medema/v37y2017i7p802-814.html
   My bibliography  Save this article

Validation of a Cardiovascular Disease Policy Microsimulation Model Using Both Survival and Receiver Operating Characteristic Curves

Author

Listed:
  • Ankur Pandya
  • Stephen Sy
  • Sylvia Cho
  • Sartaj Alam
  • Milton C. Weinstein
  • Thomas A. Gaziano

Abstract

Background. Despite some advances, cardiovascular disease (CVD) remains the leading cause of death and healthcare costs in the United States. We therefore developed a comprehensive CVD policy simulation model that identifies cost-effective approaches for reducing CVD burden. This paper aims to: 1) describe our model in detail; and 2) perform model validation analyses. Methods. The model simulates 1,000,000 adults (ages 35 to 80 years) using a variety of CVD-related epidemiological data, including previously calibrated Framingham-based risk scores for coronary heart disease and stroke. We validated our microsimulation model using recent National Health and Nutrition Examination Survey (NHANES) data, with baseline values collected in 1999-2000 and cause-specific mortality follow-up through 2011. Model-based (simulated) results were compared to observed all-cause and CVD-specific mortality data (from NHANES) for the same starting population using survival curves and, in a method not typically used for disease model validation, receiver operating characteristic (ROC) curves. Results. Observed 10-year all-cause mortality in NHANES v. the simulation model was 11.2% (95% CI, 10.3% to 12.2%) v. 10.9%; corresponding results for CVD mortality were 2.2% (1.8% to 2.7%) v. 2.6%. Areas under the ROC curves for model-predicted 10-year all-cause and CVD mortality risks were 0.83 (0.81 to 0.85) and 0.84 (0.81 to 0.88), respectively; corresponding results for 5-year risks were 0.80 (0.77 to 0.83) and 0.81 (0.75 to 0.87), respectively. Limitations. The model is limited by the uncertainties in the data used to estimate its input parameters. Additionally, our validation analyses did not include non-fatal CVD outcomes due to NHANES data limitations. Conclusions. The simulation model performed well in matching to observed nationally representative longitudinal mortality data. ROC curve analysis, which has been traditionally used for risk prediction models, can also be used to assess discrimination for disease simulation models.

Suggested Citation

  • Ankur Pandya & Stephen Sy & Sylvia Cho & Sartaj Alam & Milton C. Weinstein & Thomas A. Gaziano, 2017. "Validation of a Cardiovascular Disease Policy Microsimulation Model Using Both Survival and Receiver Operating Characteristic Curves," Medical Decision Making, , vol. 37(7), pages 802-814, October.
    • Handle: RePEc:sae:medema:v:37:y:2017:i:7:p:802-814
    DOI: 10.1177/0272989X17706081
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0272989X17706081
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0272989X17706081?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Natasha Stout & Sue Goldie, 2008. "Keeping the noise down: common random numbers for disease simulation modeling," Health Care Management Science, Springer, vol. 11(4), pages 399-406, December.
    2. Weinstein, M.C. & Coxson, P.G. & Williams, L.W. & Pass, T.M. & Stason, W.B. & Goldman, L., 1987. "Forecasting coronary heart disease incidence, mortality, and cost: The coronary heart disease policy model," American Journal of Public Health, American Public Health Association, vol. 77(11), pages 1417-1426.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Duncan Ermini Leaf & Bryan Tysinger & Dana P. Goldman & Darius N. Lakdawalla, 2021. "Predicting quantity and quality of life with the Future Elderly Model," Health Economics, John Wiley & Sons, Ltd., vol. 30(S1), pages 52-79, November.
    2. David D. Kim & Lu Wang & Brianna N. Lauren & Junxiu Liu & Matti Marklund & Yujin Lee & Renata Micha & Dariush Mozaffarian & John B. Wong, 2023. "Development and Validation of the US Diabetes, Obesity, Cardiovascular Disease Microsimulation (DOC-M) Model: Health Disparity and Economic Impact Model," Medical Decision Making, , vol. 43(7-8), pages 930-948, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. William S. Weintraub & Samuel S. Gidding, 2016. "PCSK9 Inhibitors: A Technology Worth Paying For?," PharmacoEconomics, Springer, vol. 34(3), pages 217-220, March.
    2. González-Parra, Gilberto & Villanueva-Oller, Javier & Navarro-González, F.J. & Ceberio, Josu & Luebben, Giulia, 2024. "A network-based model to assess vaccination strategies for the COVID-19 pandemic by using Bayesian optimization," Chaos, Solitons & Fractals, Elsevier, vol. 181(C).
    3. Michael F. Drummond;Adrian Towse, 1998. "From Efficacy to Cost-Effectiveness," Briefing 000438, Office of Health Economics.
    4. Yizhe Xu & Tom H. Greene & Adam P. Bress & Brian C. Sauer & Brandon K. Bellows & Yue Zhang & William S. Weintraub & Andrew E. Moran & Jincheng Shen, 2022. "Estimating the optimal individualized treatment rule from a cost‐effectiveness perspective," Biometrics, The International Biometric Society, vol. 78(1), pages 337-351, March.
    5. Mark Roberts & Louise B. Russell & A. David Paltiel & Michael Chambers & Phil McEwan & Murray Krahn, 2012. "Conceptualizing a Model," Medical Decision Making, , vol. 32(5), pages 678-689, September.
    6. Gary A. Zarkin & Laura J. Dunlap & Katherine A. Hicks & Daniel Mamo, 2005. "Benefits and costs of methadone treatment: results from a lifetime simulation model," Health Economics, John Wiley & Sons, Ltd., vol. 14(11), pages 1133-1150, November.
    7. Meltzer, David, 1997. "Accounting for future costs in medical cost-effectiveness analysis," Journal of Health Economics, Elsevier, vol. 16(1), pages 33-64, February.
    8. Troost, Christian & Huber, Robert & Bell, Andrew R. & van Delden, Hedwig & Filatova, Tatiana & Le, Quang Bao & Lippe, Melvin & Niamir, Leila & Polhill, J. Gareth & Sun, Zhanli & Berger, Thomas, 2023. "How to keep it adequate: A protocol for ensuring validity in agent-based simulation," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 159, pages 1-21.
    9. repec:ijm:journl:v109:y:2017:i:1:p:167-200 is not listed on IDEAS
    10. K Cooper & S C Brailsford & R Davies, 2007. "Choice of modelling technique for evaluating health care interventions," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 58(2), pages 168-176, February.
    11. Tekeshe A Mekonnen & Michelle C Odden & Pamela G Coxson & David Guzman & James Lightwood & Y Claire Wang & Kirsten Bibbins-Domingo, 2013. "Health Benefits of Reducing Sugar-Sweetened Beverage Intake in High Risk Populations of California: Results from the Cardiovascular Disease (CVD) Policy Model," PLOS ONE, Public Library of Science, vol. 8(12), pages 1-9, December.
    12. Karen T. Hicklin & Julie S. Ivy & James R. Wilson & Fay Cobb Payton & Meera Viswanathan & Evan R. Myers, 2019. "Simulation model of the relationship between cesarean section rates and labor duration," Health Care Management Science, Springer, vol. 22(4), pages 635-657, December.
    13. Davies, Ruth & Roderick, Paul & Raftery, James, 2003. "The evaluation of disease prevention and treatment using simulation models," European Journal of Operational Research, Elsevier, vol. 150(1), pages 53-66, October.
    14. K. Cooper & S. Brailsford & R. Davies & J. Raftery, 2006. "A review of health care models for coronary heart disease interventions," Health Care Management Science, Springer, vol. 9(4), pages 311-324, November.
    15. William Weintraub & Samuel Gidding, 2016. "PCSK9 Inhibitors: A Technology Worth Paying For?," PharmacoEconomics, Springer, vol. 34(3), pages 217-220, March.
    16. James E. Smith & Ralph L. Keeney, 2005. "Your Money or Your Life: A Prescriptive Model for Health, Safety, and Consumption Decisions," Management Science, INFORMS, vol. 51(9), pages 1309-1325, September.
    17. Amy O’Sullivan & Jaime Rubin & Joshua Nyambose & Andreas Kuznik & David Cohen & David Thompson, 2011. "Cost Estimation of Cardiovascular Disease Events in the US," PharmacoEconomics, Springer, vol. 29(8), pages 693-704, August.
    18. Uwe Siebert & Oguzhan Alagoz & Ahmed M. Bayoumi & Beate Jahn & Douglas K. Owens & David J. Cohen & Karen M. Kuntz, 2012. "State-Transition Modeling," Medical Decision Making, , vol. 32(5), pages 690-700, September.
    19. Ian Philips & Graham Clarke & David Watling, 2017. "A Fine Grained Hybrid Spatial Microsimulation Technique for Generating Detailed Synthetic Individuals from Multiple Data Sources: An Application To Walking And Cycling," International Journal of Microsimulation, International Microsimulation Association, vol. 10(1), pages 167-200.
    20. Uwe Siebert, 2003. "When should decision-analytic modeling be used in the economic evaluation of health care?," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 4(3), pages 143-150, September.
    21. Peter Lindgren & Bengt Jönsson, 2012. "Cost–effectiveness of statins revisited: lessons learned about the value of innovation," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 13(4), pages 445-450, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:medema:v:37:y:2017:i:7:p:802-814. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.