IDEAS home Printed from https://ideas.repec.org/a/sae/medema/v38y2018i2p212-224.html
   My bibliography  Save this article

Using Observational Data to Calibrate Simulation Models

Author

Listed:
  • Eleanor J. Murray
  • James M. Robins
  • George R. Seage III
  • Sara Lodi
  • Emily P. Hyle
  • Krishna P. Reddy
  • Kenneth A. Freedberg
  • Miguel A. Hernán

Abstract

Background. Individual-level simulation models are valuable tools for comparing the impact of clinical or public health interventions on population health and cost outcomes over time. However, a key challenge is ensuring that outcome estimates correctly reflect real-world impacts. Calibration to targets obtained from randomized trials may be insufficient if trials do not exist for populations, time periods, or interventions of interest. Observational data can provide a wider range of calibration targets but requires methods to adjust for treatment-confounder feedback. We propose the use of the parametric g-formula to estimate calibration targets and present a case-study to demonstrate its application. Methods. We used the parametric g-formula applied to data from the HIV-CAUSAL Collaboration to estimate calibration targets for 7-y risks of AIDS and/or death (AIDS/death), as defined by the Center for Disease Control and Prevention under 3 treatment initiation strategies. We compared these targets to projections from the Cost-effectiveness of Preventing AIDS Complications (CEPAC) model for treatment-naïve individuals presenting to care in the following year ranges: 1996 to 1999, 2000 to 2002, or 2003 onwards. Results. The parametric g-formula estimated a decreased risk of AIDS/death over time and with earlier treatment. The uncalibrated CEPAC model successfully reproduced targets obtained via the g-formula for baseline 1996 to 1999, but over-estimated calibration targets in contemporary populations and failed to reproduce time trends in AIDS/death risk. Calibration to g-formula targets improved CEPAC model fit for contemporary populations. Conclusion. Individual-level simulation models are developed based on best available information about disease processes in one or more populations of interest, but these processes can change over time or between populations. The parametric g-formula provides a method for using observational data to obtain valid calibration targets and enables updating of simulation model inputs when randomized trials are not available.

Suggested Citation

  • Eleanor J. Murray & James M. Robins & George R. Seage III & Sara Lodi & Emily P. Hyle & Krishna P. Reddy & Kenneth A. Freedberg & Miguel A. Hernán, 2018. "Using Observational Data to Calibrate Simulation Models," Medical Decision Making, , vol. 38(2), pages 212-224, February.
    • Handle: RePEc:sae:medema:v:38:y:2018:i:2:p:212-224
    DOI: 10.1177/0272989X17738753
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0272989X17738753
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0272989X17738753?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Jakob Grazzini, 2011. "Consistent Estimation of Agent Based Models," LABORatorio R. Revelli Working Papers Series 110, LABORatorio R. Revelli, Centre for Employment Studies.
    2. Rutter, Carolyn M. & Miglioretti, Diana L. & Savarino, James E., 2009. "Bayesian Calibration of Microsimulation Models," Journal of the American Statistical Association, American Statistical Association, vol. 104(488), pages 1338-1350.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. André Moser & Milo A. Puhan & Marcel Zwahlen, 2020. "The role of causal inference in health services research II: a framework for causal inference," International Journal of Public Health, Springer;Swiss School of Public Health (SSPH+), vol. 65(3), pages 367-370, April.
    2. C Marijn Hazelbag & Jonathan Dushoff & Emanuel M Dominic & Zinhle E Mthombothi & Wim Delva, 2020. "Calibration of individual-based models to epidemiological data: A systematic review," PLOS Computational Biology, Public Library of Science, vol. 16(5), pages 1-17, May.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jing Voon Chen & Julia L. Higle & Michael Hintlian, 2018. "A systematic approach for examining the impact of calibration uncertainty in disease modeling," Computational Management Science, Springer, vol. 15(3), pages 541-561, October.
    2. Douglas Taylor & Vivek Pawar & Denise Kruzikas & Kristen Gilmore & Myrlene Sanon & Milton Weinstein, 2012. "Incorporating Calibrated Model Parameters into Sensitivity Analyses," PharmacoEconomics, Springer, vol. 30(2), pages 119-126, February.
    3. Grazzini, J., 2011. "Experimental Based, Agent Based Stock Market," CeNDEF Working Papers 11-07, Universiteit van Amsterdam, Center for Nonlinear Dynamics in Economics and Finance.
    4. Sophie Whyte & Cathal Walsh & Jim Chilcott, 2011. "Bayesian Calibration of a Natural History Model with Application to a Population Model for Colorectal Cancer," Medical Decision Making, , vol. 31(4), pages 625-641, July.
    5. Alex van der Steen & Joost van Rosmalen & Sonja Kroep & Frank van Hees & Ewout W. Steyerberg & Harry J. de Koning & Marjolein van Ballegooijen & Iris Lansdorp-Vogelaar, 2016. "Calibrating Parameters for Microsimulation Disease Models," Medical Decision Making, , vol. 36(5), pages 652-665, July.
    6. Maria DeYoreo & Iris Lansdorp-Vogelaar & Amy B. Knudsen & Karen M. Kuntz & Ann G. Zauber & Carolyn M. Rutter, 2020. "Validation of Colorectal Cancer Models on Long-term Outcomes from a Randomized Controlled Trial," Medical Decision Making, , vol. 40(8), pages 1034-1040, November.
    7. Vahab Vahdat & Oguzhan Alagoz & Jing Voon Chen & Leila Saoud & Bijan J. Borah & Paul J. Limburg, 2023. "Calibration and Validation of the Colorectal Cancer and Adenoma Incidence and Mortality (CRC-AIM) Microsimulation Model Using Deep Neural Networks," Medical Decision Making, , vol. 43(6), pages 719-736, August.
    8. Grazzini Jakob, 2011. "Estimating Micromotives from Macrobehavior," Department of Economics and Statistics Cognetti de Martiis. Working Papers 201111, University of Turin.
    9. Arias Chao, Manuel & Kulkarni, Chetan & Goebel, Kai & Fink, Olga, 2022. "Fusing physics-based and deep learning models for prognostics," Reliability Engineering and System Safety, Elsevier, vol. 217(C).
    10. Stavroula A Chrysanthopoulou, 2017. "MILC: A Microsimulation Model of the Natural History of Lung Cancer," International Journal of Microsimulation, International Microsimulation Association, vol. 10(3), pages 5-26.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:medema:v:38:y:2018:i:2:p:212-224. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.