IDEAS home Printed from https://ideas.repec.org/a/eee/reensy/v204y2020ics0951832020306426.html
   My bibliography  Save this article

A generalized Bayesian approach to model calibration

Author

Listed:
  • Tohme, Tony
  • Vanslette, Kevin
  • Youcef-Toumi, Kamal

Abstract

In model development, model calibration and validation play complementary roles toward learning reliable models. In this article, we expand the Bayesian Validation Metric framework to a general calibration and validation framework by inverting the validation mathematics into a generalized Bayesian method for model calibration and regression. We perform Bayesian regression based on a user’s definition of model-data agreement. This allows for model selection on any type of data distribution, unlike Bayesian and standard regression techniques, that “fail†in some cases. We show that our tool is capable of representing and combining least squares, likelihood-based, and Bayesian calibration techniques in a single framework while being able to generalize aspects of these methods. This tool also offers new insights into the interpretation of the predictive envelopes (also known as confidence bands) while giving the analyst more control over these envelopes. We demonstrate the validity of our method by providing three numerical examples to calibrate different models, including a model for energy dissipation in lap joints under impact loading. By calibrating models with respect to the validation metrics one desires a model to ultimately pass, reliability and safety metrics may be integrated into and automatically adopted by the model in the calibration phase.

Suggested Citation

  • Tohme, Tony & Vanslette, Kevin & Youcef-Toumi, Kamal, 2020. "A generalized Bayesian approach to model calibration," Reliability Engineering and System Safety, Elsevier, vol. 204(C).
  • Handle: RePEc:eee:reensy:v:204:y:2020:i:c:s0951832020306426
    DOI: 10.1016/j.ress.2020.107141
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0951832020306426
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.ress.2020.107141?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Sankararaman, Shankar & Mahadevan, Sankaran, 2011. "Model validation under epistemic uncertainty," Reliability Engineering and System Safety, Elsevier, vol. 96(9), pages 1232-1241.
    2. Zio, E., 2018. "The future of risk assessment," Reliability Engineering and System Safety, Elsevier, vol. 177(C), pages 176-190.
    3. Sankararaman, Shankar & Mahadevan, Sankaran, 2015. "Integration of model verification, validation, and calibration for uncertainty quantification in engineering systems," Reliability Engineering and System Safety, Elsevier, vol. 138(C), pages 194-209.
    4. Raillon, L. & Ghiaus, C., 2018. "An efficient Bayesian experimental calibration of dynamic thermal models," Energy, Elsevier, vol. 152(C), pages 818-833.
    5. Vanslette, Kevin & Tohme, Tony & Youcef-Toumi, Kamal, 2020. "A general model validation and testing tool," Reliability Engineering and System Safety, Elsevier, vol. 195(C).
    6. VanDerHorn, Eric & Mahadevan, Sankaran, 2018. "Bayesian model updating with summarized statistical and reliability data," Reliability Engineering and System Safety, Elsevier, vol. 172(C), pages 12-24.
    7. Nannapaneni, Saideep & Mahadevan, Sankaran, 2016. "Reliability analysis under epistemic uncertainty," Reliability Engineering and System Safety, Elsevier, vol. 155(C), pages 9-20.
    8. Wu, Danqing & Lu, Zhenzhou & Wang, Yanping & Cheng, Lei, 2015. "Model validation and calibration based on component functions of model output," Reliability Engineering and System Safety, Elsevier, vol. 140(C), pages 59-70.
    9. Penttinen, Jussi-Pekka & Niemi, Arto & Gutleber, Johannes & Koskinen, Kari T. & Coatanéa, Eric & Laitinen, Jouko, 2019. "An open modelling approach for availability and reliability of systems," Reliability Engineering and System Safety, Elsevier, vol. 183(C), pages 387-399.
    10. Mullins, Joshua & Ling, You & Mahadevan, Sankaran & Sun, Lin & Strachan, Alejandro, 2016. "Separation of aleatory and epistemic uncertainty in probabilistic model validation," Reliability Engineering and System Safety, Elsevier, vol. 147(C), pages 49-59.
    11. Wang, Chong & Matthies, Hermann G., 2019. "Novel model calibration method via non-probabilistic interval characterization and Bayesian theory," Reliability Engineering and System Safety, Elsevier, vol. 183(C), pages 84-92.
    12. Zio, Enrico, 2016. "Challenges in the vulnerability and risk analysis of critical infrastructures," Reliability Engineering and System Safety, Elsevier, vol. 152(C), pages 137-150.
    13. repec:dau:papers:123456789/5724 is not listed on IDEAS
    14. Rebba, Ramesh & Mahadevan, Sankaran, 2008. "Computational methods for model reliability assessment," Reliability Engineering and System Safety, Elsevier, vol. 93(8), pages 1197-1207.
    15. Sankararaman, Shankar & Mahadevan, Sankaran, 2011. "Likelihood-based representation of epistemic uncertainty due to sparse point data and/or interval data," Reliability Engineering and System Safety, Elsevier, vol. 96(7), pages 814-824.
    16. Trucano, T.G. & Swiler, L.P. & Igusa, T. & Oberkampf, W.L. & Pilch, M., 2006. "Calibration, validation, and sensitivity analysis: What's what," Reliability Engineering and System Safety, Elsevier, vol. 91(10), pages 1331-1357.
    17. Park, Inseok & Amarchinta, Hemanth K. & Grandhi, Ramana V., 2010. "A Bayesian approach for quantification of model uncertainty," Reliability Engineering and System Safety, Elsevier, vol. 95(7), pages 777-785.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Tohme, Tony & Vanslette, Kevin & Youcef-Toumi, Kamal, 2023. "Reliable neural networks for regression uncertainty estimation," Reliability Engineering and System Safety, Elsevier, vol. 229(C).
    2. Jung, Yongsu & Jo, Hwisang & Choo, Jeonghwan & Lee, Ikjin, 2022. "Statistical model calibration and design optimization under aleatory and epistemic uncertainty," Reliability Engineering and System Safety, Elsevier, vol. 222(C).
    3. Tang, Cong & He, Shu-Yu & Zhou, Wan-Huan, 2022. "Settlement-based framework for long-term serviceability assessment of immersed tunnels," Reliability Engineering and System Safety, Elsevier, vol. 228(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Vanslette, Kevin & Tohme, Tony & Youcef-Toumi, Kamal, 2020. "A general model validation and testing tool," Reliability Engineering and System Safety, Elsevier, vol. 195(C).
    2. Sankararaman, Shankar & Mahadevan, Sankaran, 2015. "Integration of model verification, validation, and calibration for uncertainty quantification in engineering systems," Reliability Engineering and System Safety, Elsevier, vol. 138(C), pages 194-209.
    3. Wang, Chong & Matthies, Hermann G., 2019. "Novel model calibration method via non-probabilistic interval characterization and Bayesian theory," Reliability Engineering and System Safety, Elsevier, vol. 183(C), pages 84-92.
    4. Ling, You & Mahadevan, Sankaran, 2013. "Quantitative model validation techniques: New insights," Reliability Engineering and System Safety, Elsevier, vol. 111(C), pages 217-231.
    5. Jung, Yongsu & Jo, Hwisang & Choo, Jeonghwan & Lee, Ikjin, 2022. "Statistical model calibration and design optimization under aleatory and epistemic uncertainty," Reliability Engineering and System Safety, Elsevier, vol. 222(C).
    6. Khastgir, Siddartha & Brewerton, Simon & Thomas, John & Jennings, Paul, 2021. "Systems Approach to Creating Test Scenarios for Automated Driving Systems," Reliability Engineering and System Safety, Elsevier, vol. 215(C).
    7. Mi, Jinhua & Lu, Ning & Li, Yan-Feng & Huang, Hong-Zhong & Bai, Libing, 2022. "An evidential network-based hierarchical method for system reliability analysis with common cause failures and mixed uncertainties," Reliability Engineering and System Safety, Elsevier, vol. 220(C).
    8. Perrin, G., 2020. "Adaptive calibration of a computer code with time-series output," Reliability Engineering and System Safety, Elsevier, vol. 196(C).
    9. Ao, Dan & Hu, Zhen & Mahadevan, Sankaran, 2017. "Design of validation experiments for life prediction models," Reliability Engineering and System Safety, Elsevier, vol. 165(C), pages 22-33.
    10. Sankararaman, S. & Mahadevan, S., 2013. "Separating the contributions of variability and parameter uncertainty in probability distributions," Reliability Engineering and System Safety, Elsevier, vol. 112(C), pages 187-199.
    11. Hao, Zhaojun & Di Maio, Francesco & Zio, Enrico, 2023. "A sequential decision problem formulation and deep reinforcement learning solution of the optimization of O&M of cyber-physical energy systems (CPESs) for reliable and safe power production and supply," Reliability Engineering and System Safety, Elsevier, vol. 235(C).
    12. Wang, Yuhao & Liu, Yongming, 2020. "Bayesian entropy network for fusion of different types of information," Reliability Engineering and System Safety, Elsevier, vol. 195(C).
    13. Morales-Torres, Adrián & Escuder-Bueno, Ignacio & Serrano-Lombillo, Armando & Castillo Rodríguez, Jesica T., 2019. "Dealing with epistemic uncertainty in risk-informed decision making for dam safety management," Reliability Engineering and System Safety, Elsevier, vol. 191(C).
    14. Hund, Lauren & Schroeder, Benjamin & Rumsey, Kellin & Huerta, Gabriel, 2018. "Distinguishing between model- and data-driven inferences for high reliability statistical predictions," Reliability Engineering and System Safety, Elsevier, vol. 180(C), pages 201-210.
    15. Terje Aven, 2019. "The Call for a Shift from Risk to Resilience: What Does it Mean?," Risk Analysis, John Wiley & Sons, vol. 39(6), pages 1196-1203, June.
    16. Robinson, Allen C. & Drake, Richard R. & Swan, M. Scot & Bennett, Nichelle L. & Smith, Thomas M. & Hooper, Russell & Laity, George R., 2021. "A software environment for effective reliability management for pulsed power design," Reliability Engineering and System Safety, Elsevier, vol. 211(C).
    17. Kim, Wongon & Yoon, Heonjun & Lee, Guesuk & Kim, Taejin & Youn, Byeng D., 2020. "A new calibration metric that considers statistical correlation: Marginal Probability and Correlation Residuals," Reliability Engineering and System Safety, Elsevier, vol. 195(C).
    18. Wei Wang & Francesco Di Maio & Enrico Zio, 2019. "Adversarial Risk Analysis to Allocate Optimal Defense Resources for Protecting Cyber–Physical Systems from Cyber Attacks," Risk Analysis, John Wiley & Sons, vol. 39(12), pages 2766-2785, December.
    19. Wang, Zhenqiang & Jia, Gaofeng, 2020. "Augmented sample-based approach for efficient evaluation of risk sensitivity with respect to epistemic uncertainty in distribution parameters," Reliability Engineering and System Safety, Elsevier, vol. 197(C).
    20. Park, Inseok & Grandhi, Ramana V., 2014. "A Bayesian statistical method for quantifying model form uncertainty and two model combination methods," Reliability Engineering and System Safety, Elsevier, vol. 129(C), pages 46-56.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:reensy:v:204:y:2020:i:c:s0951832020306426. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/reliability-engineering-and-system-safety .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.