IDEAS home Printed from https://ideas.repec.org/a/sae/medema/v38y2018i7p810-821.html
   My bibliography  Save this article

Nonidentifiability in Model Calibration and Implications for Medical Decision Making

Author

Listed:
  • Fernando Alarid-Escudero

    (Division of Health Policy and Management, University of Minnesota School of Public Health, Minneapolis, MN)

  • Richard F. MacLehose

    (Division of Epidemiology and Community Health, University of Minnesota School of Public Health, Minneapolis, MN)

  • Yadira Peralta

    (Department of Educational Psychology, University of Minnesota, Minneapolis, MN)

  • Karen M. Kuntz

    (Division of Health Policy and Management, University of Minnesota School of Public Health, Minneapolis, MN)

  • Eva A. Enns

    (Division of Health Policy and Management, University of Minnesota School of Public Health, Minneapolis, MN)

Abstract

Background. Calibration is the process of estimating parameters of a mathematical model by matching model outputs to calibration targets. In the presence of nonidentifiability, multiple parameter sets solve the calibration problem, which may have important implications for decision making. We evaluate the implications of nonidentifiability on the optimal strategy and provide methods to check for nonidentifiability. Methods. We illustrate nonidentifiability by calibrating a 3-state Markov model of cancer relative survival (RS). We performed 2 different calibration exercises: 1) only including RS as a calibration target and 2) adding the ratio between the 2 nondeath states over time as an additional target. We used the Nelder-Mead (NM) algorithm to identify parameter sets that best matched the calibration targets. We used collinearity and likelihood profile analyses to check for nonidentifiability. We then estimated the benefit of a hypothetical treatment in terms of life expectancy gains using different, but equally good-fitting, parameter sets. We also applied collinearity analysis to a realistic model of the natural history of colorectal cancer. Results. When only RS is used as the calibration target, 2 different parameter sets yield similar maximum likelihood values. The high collinearity index and the bimodal likelihood profile on both parameters demonstrated the presence of nonidentifiability. These different, equally good-fitting parameter sets produce different estimates of the treatment effectiveness (0.67 v. 0.31 years), which could influence the optimal decision. By incorporating the additional target, the model becomes identifiable with a collinearity index of 3.5 and a unimodal likelihood profile. Conclusions. In the presence of nonidentifiability, equally likely parameter estimates might yield different conclusions. Checking for the existence of nonidentifiability and its implications should be incorporated into standard model calibration procedures.

Suggested Citation

  • Fernando Alarid-Escudero & Richard F. MacLehose & Yadira Peralta & Karen M. Kuntz & Eva A. Enns, 2018. "Nonidentifiability in Model Calibration and Implications for Medical Decision Making," Medical Decision Making, , vol. 38(7), pages 810-821, October.
    • Handle: RePEc:sae:medema:v:38:y:2018:i:7:p:810-821
    DOI: 10.1177/0272989X18792283
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0272989X18792283
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0272989X18792283?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Jack P.C. Kleijnen, 2015. "Design and Analysis of Simulation Experiments," International Series in Operations Research and Management Science, Springer, edition 2, number 978-3-319-18087-8, April.
    2. Alan Brennan & Stephen E. Chick & Ruth Davies, 2006. "A taxonomy of model structures for economic evaluation of health technologies," Health Economics, John Wiley & Sons, Ltd., vol. 15(12), pages 1295-1310, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Vahab Vahdat & Oguzhan Alagoz & Jing Voon Chen & Leila Saoud & Bijan J. Borah & Paul J. Limburg, 2023. "Calibration and Validation of the Colorectal Cancer and Adenoma Incidence and Mortality (CRC-AIM) Microsimulation Model Using Deep Neural Networks," Medical Decision Making, , vol. 43(6), pages 719-736, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wen Shi & Xi Chen & Jennifer Shang, 2019. "An Efficient Morris Method-Based Framework for Simulation Factor Screening," INFORMS Journal on Computing, INFORMS, vol. 31(4), pages 745-770, October.
    2. Tian, Wei & Song, Jitian & Li, Zhanyong & de Wilde, Pieter, 2014. "Bootstrap techniques for sensitivity analysis and model selection in building thermal performance analysis," Applied Energy, Elsevier, vol. 135(C), pages 320-328.
    3. Alfieri, Arianna & Matta, Andrea, 2012. "Mathematical programming formulations for approximate simulation of multistage production systems," European Journal of Operational Research, Elsevier, vol. 219(3), pages 773-783.
    4. Bernhard Ultsch & Oliver Damm & Philippe Beutels & Joke Bilcke & Bernd Brüggenjürgen & Andreas Gerber-Grote & Wolfgang Greiner & Germaine Hanquet & Raymond Hutubessy & Mark Jit & Mirjam Knol & Rüdiger, 2016. "Methods for Health Economic Evaluation of Vaccines and Immunization Decision Frameworks: A Consensus Framework from a European Vaccine Economics Community," PharmacoEconomics, Springer, vol. 34(3), pages 227-244, March.
    5. Becky Pennington & Alex Filby & Lesley Owen & Matthew Taylor, 2018. "Smoking Cessation: A Comparison of Two Model Structures," PharmacoEconomics, Springer, vol. 36(9), pages 1101-1112, September.
    6. Dellino, Gabriella & Kleijnen, Jack P.C. & Meloni, Carlo, 2010. "Robust optimization in simulation: Taguchi and Response Surface Methodology," International Journal of Production Economics, Elsevier, vol. 125(1), pages 52-59, May.
    7. Peter J. Dodd & Jeff J. Pennington & Liza Bronner Murrison & David W. Dowdy, 2018. "Simple Inclusion of Complex Diagnostic Algorithms in Infectious Disease Models for Economic Evaluation," Medical Decision Making, , vol. 38(8), pages 930-941, November.
    8. Angun, M.E. & Kleijnen, Jack P.C., 2012. "An asymptotic test of optimality conditions in multiresponse simulation optimization," Other publications TiSEM a69dfa59-b0e1-45bd-8cd6-a, Tilburg University, School of Economics and Management.
    9. Jonathan Karnon & James Stahl & Alan Brennan & J. Jaime Caro & Javier Mar & Jörgen Möller, 2012. "Modeling Using Discrete Event Simulation," Medical Decision Making, , vol. 32(5), pages 701-711, September.
    10. Jack P. C. Kleijnen, 2017. "Comment on Park et al.’s “Robust Kriging in computer experiments”," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 68(6), pages 739-740, June.
    11. Stuart J. Wright & William G. Newman & Katherine Payne, 2019. "Accounting for Capacity Constraints in Economic Evaluations of Precision Medicine: A Systematic Review," PharmacoEconomics, Springer, vol. 37(8), pages 1011-1027, August.
    12. Arielle Anderer & Hamsa Bastani & John Silberholz, 2022. "Adaptive Clinical Trial Designs with Surrogates: When Should We Bother?," Management Science, INFORMS, vol. 68(3), pages 1982-2002, March.
    13. Kleijnen, Jack P.C., 2009. "Kriging metamodeling in simulation: A review," European Journal of Operational Research, Elsevier, vol. 192(3), pages 707-716, February.
    14. Eren Demir & David Southern, 2017. "Enabling better management of patients: discrete event simulation combined with the STAR approach," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 68(5), pages 577-590, May.
    15. Jen Kruger & Daniel Pollard & Hasan Basarir & Praveen Thokala & Debbie Cooke & Marie Clark & Rod Bond & Simon Heller & Alan Brennan, 2015. "Incorporating Psychological Predictors of Treatment Response into Health Economic Simulation Models," Medical Decision Making, , vol. 35(7), pages 872-887, October.
    16. Kleijnen, Jack P.C. & Ridder, A.A.N. & Rubinstein, R.Y., 2010. "Variance Reduction Techniques in Monte Carlo Methods," Other publications TiSEM 87680d1a-53c1-4107-ada4-7, Tilburg University, School of Economics and Management.
    17. Sarah Bates & Thomas Bayley & Paul Norman & Penny Breeze & Alan Brennan, 2020. "A Systematic Review of Methods to Predict Weight Trajectories in Health Economic Models of Behavioral Weight-Management Programs: The Potential Role of Psychosocial Factors," Medical Decision Making, , vol. 40(1), pages 90-105, January.
    18. F. Tomini & F. Prinzen & A. D. I. Asselt, 2016. "A review of economic evaluation models for cardiac resynchronization therapy with implantable cardioverter defibrillators in patients with heart failure," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 17(9), pages 1159-1172, December.
    19. Martin, Rebecca & Lazakis, Iraklis & Barbouchi, Sami & Johanning, Lars, 2016. "Sensitivity analysis of offshore wind farm operation and maintenance cost and availability," Renewable Energy, Elsevier, vol. 85(C), pages 1226-1236.
    20. Luis Hernandez & Asli Ozen & Rodrigo DosSantos & Denis Getsios, 2016. "Systematic Review of Model-Based Economic Evaluations of Treatments for Alzheimer’s Disease," PharmacoEconomics, Springer, vol. 34(7), pages 681-707, July.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:medema:v:38:y:2018:i:7:p:810-821. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.