IDEAS home Printed from https://ideas.repec.org/a/sae/medema/v33y2013i7p880-890.html
   My bibliography  Save this article

Linear Regression Metamodeling as a Tool to Summarize and Present Simulation Model Results

Author

Listed:
  • Hawre Jalal
  • Bryan Dowd
  • François Sainfort
  • Karen M. Kuntz

Abstract

Background/Objective. Modelers lack a tool to systematically and clearly present complex model results, including those from sensitivity analyses. The objective was to propose linear regression metamodeling as a tool to increase transparency of decision analytic models and better communicate their results. Methods. We used a simplified cancer cure model to demonstrate our approach. The model computed the lifetime cost and benefit of 3 treatment options for cancer patients. We simulated 10,000 cohorts in a probabilistic sensitivity analysis (PSA) and regressed the model outcomes on the standardized input parameter values in a set of regression analyses. We used the regression coefficients to describe measures of sensitivity analyses, including threshold and parameter sensitivity analyses. We also compared the results of the PSA to deterministic full-factorial and one-factor-at-a-time designs. Results. The regression intercept represented the estimated base-case outcome, and the other coefficients described the relative parameter uncertainty in the model. We defined simple relationships that compute the average and incremental net benefit of each intervention. Metamodeling produced outputs similar to traditional deterministic 1-way or 2-way sensitivity analyses but was more reliable since it used all parameter values. Conclusions. Linear regression metamodeling is a simple, yet powerful, tool that can assist modelers in communicating model characteristics and sensitivity analyses.

Suggested Citation

  • Hawre Jalal & Bryan Dowd & François Sainfort & Karen M. Kuntz, 2013. "Linear Regression Metamodeling as a Tool to Summarize and Present Simulation Model Results," Medical Decision Making, , vol. 33(7), pages 880-890, October.
    • Handle: RePEc:sae:medema:v:33:y:2013:i:7:p:880-890
    DOI: 10.1177/0272989X13492014
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0272989X13492014
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0272989X13492014?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. M. D. Stevenson & J. Oakley & J. B. Chilcott, 2004. "Gaussian Process Modeling in Conjunction with Individual Patient Simulation Modeling: A Case Study Describing the Calculation of Cost-Effectiveness Ratios for the Treatment of Established Osteoporosis," Medical Decision Making, , vol. 24(1), pages 89-100, January.
    2. Kleijnen, Jack P. C., 2005. "An overview of the design and analysis of simulation experiments for sensitivity analysis," European Journal of Operational Research, Elsevier, vol. 164(2), pages 287-300, July.
    3. Kleijnen, J.P.C. & Sargent, R., 1997. "A Methodology for Fitting and Validating Metamodels in Simulation," Other publications TiSEM b8c2217c-f02c-4786-b704-7, Tilburg University, School of Economics and Management.
    4. Doug Coyle & Jeremy Oakley, 2008. "Estimating the expected value of partial perfect information: a review of methods," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 9(3), pages 251-259, August.
    5. Jack P.C. Kleijnen, 2015. "Design and Analysis of Simulation Experiments," International Series in Operations Research and Management Science, Springer, edition 2, number 978-3-319-18087-8, July-Dece.
    6. Kleijnen, Jack P. C. & Sargent, Robert G., 2000. "A methodology for fitting and validating metamodels in simulation," European Journal of Operational Research, Elsevier, vol. 120(1), pages 14-29, January.
    7. Robert W. Blanning, 1974. "The Sources and Uses of Sensitivity Information," Interfaces, INFORMS, vol. 4(4), pages 32-38, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. David Glynn & Susan Griffin & Nils Gutacker & Simon Walker, 2024. "Methods to Quantify the Importance of Parameters for Model Updating and Distributional Adaptation," Medical Decision Making, , vol. 44(7), pages 802-810, October.
    2. Radaideh, Majdi I. & Kozlowski, Tomasz, 2020. "Surrogate modeling of advanced computer simulations using deep Gaussian processes," Reliability Engineering and System Safety, Elsevier, vol. 195(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Poropudas, Jirka & Virtanen, Kai, 2011. "Simulation metamodeling with dynamic Bayesian networks," European Journal of Operational Research, Elsevier, vol. 214(3), pages 644-655, November.
    2. Jack P. C. Kleijnen & Susan M. Sanchez & Thomas W. Lucas & Thomas M. Cioppa, 2005. "State-of-the-Art Review: A User’s Guide to the Brave New World of Designing Simulation Experiments," INFORMS Journal on Computing, INFORMS, vol. 17(3), pages 263-289, August.
    3. Kleijnen, Jack P.C., 2009. "Kriging metamodeling in simulation: A review," European Journal of Operational Research, Elsevier, vol. 192(3), pages 707-716, February.
    4. Strang, Kenneth David, 2012. "Importance of verifying queue model assumptions before planning with simulation software," European Journal of Operational Research, Elsevier, vol. 218(2), pages 493-504.
    5. Jin, Ding & Hedtrich, Johannes & Henning, Christian, 2018. "Applying Meta modeling for extended CGE-modeling: Sample techniques and potential application," Conference papers 332947, Purdue University, Center for Global Trade Analysis, Global Trade Analysis Project.
    6. Marrel, Amandine & Iooss, Bertrand & Van Dorpe, François & Volkova, Elena, 2008. "An efficient methodology for modeling complex computer codes with Gaussian processes," Computational Statistics & Data Analysis, Elsevier, vol. 52(10), pages 4731-4744, June.
    7. Michael C. Fu & Huashuai Qu, 2014. "Regression Models Augmented with Direct Stochastic Gradient Estimators," INFORMS Journal on Computing, INFORMS, vol. 26(3), pages 484-499, August.
    8. Imry Rosenbaum & Jeremy Staum, 2017. "Multilevel Monte Carlo Metamodeling," Operations Research, INFORMS, vol. 65(4), pages 1062-1077, August.
    9. Scott L. Rosen & Christopher P. Saunders & Samar K Guharay, 2015. "A Structured Approach for Rapidly Mapping Multilevel System Measures via Simulation Metamodeling," Systems Engineering, John Wiley & Sons, vol. 18(1), pages 87-101, January.
    10. Shi, Wen & Shang, Jennifer & Liu, Zhixue & Zuo, Xiaolu, 2014. "Optimal design of the auto parts supply chain for JIT operations: Sequential bifurcation factor screening and multi-response surface methodology," European Journal of Operational Research, Elsevier, vol. 236(2), pages 664-676.
    11. Happe, Kathrin & Kellermann, Konrad & Balmann, Alfons, 2006. "Agent-based analysis of agricultural policies: An illustration of the agricultural policy simulator AgriPoliS, its adaptation and behavior," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 11(1).
    12. Kleijnen, Jack P.C. & Mehdad, E., 2012. "Kriging in Multi-response Simulation, including a Monte Carlo Laboratory (Replaced by 2014-012)," Discussion Paper 2012-039, Tilburg University, Center for Economic Research.
    13. Reis dos Santos, M. Isabel & Porta Nova, Acacio M.O., 2006. "Statistical fitting and validation of non-linear simulation metamodels: A case study," European Journal of Operational Research, Elsevier, vol. 171(1), pages 53-63, May.
    14. Kleijnen, Jack P.C. & Mehdad, E., 2012. "Kriging in Multi-response Simulation, including a Monte Carlo Laboratory (Replaced by 2014-012)," Other publications TiSEM cf311469-5f8c-4c1e-ad4f-6, Tilburg University, School of Economics and Management.
    15. Iooss, Bertrand & Van Dorpe, François & Devictor, Nicolas, 2006. "Response surfaces and sensitivity analyses for an environmental model of dose calculations," Reliability Engineering and System Safety, Elsevier, vol. 91(10), pages 1241-1251.
    16. Katarzyna Growiec & Jakub Growiec & Bogumil Kaminski, 2017. "Social Network Structure and The Trade-Off Between Social Utility and Economic Performance," KAE Working Papers 2017-026, Warsaw School of Economics, Collegium of Economic Analysis.
    17. Acharki, Naoufal & Bertoncello, Antoine & Garnier, Josselin, 2023. "Robust prediction interval estimation for Gaussian processes by cross-validation method," Computational Statistics & Data Analysis, Elsevier, vol. 178(C).
    18. Giorgio Fagiolo & Mattia Guerini & Francesco Lamperti & Alessio Moneta & Andrea Roventini, 2017. "Validation of Agent-Based Models in Economics and Finance," LEM Papers Series 2017/23, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    19. H. Christopher Frey & Sumeet R. Patil, 2002. "Identification and Review of Sensitivity Analysis Methods," Risk Analysis, John Wiley & Sons, vol. 22(3), pages 553-578, June.
    20. Tunali, S. & Batmaz, I., 2003. "A metamodeling methodology involving both qualitative and quantitative input factors," European Journal of Operational Research, Elsevier, vol. 150(2), pages 437-450, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:medema:v:33:y:2013:i:7:p:880-890. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.