IDEAS home Printed from https://ideas.repec.org/a/eee/reensy/v195y2020ics0951832019301334.html
   My bibliography  Save this article

A new calibration metric that considers statistical correlation: Marginal Probability and Correlation Residuals

Author

Listed:
  • Kim, Wongon
  • Yoon, Heonjun
  • Lee, Guesuk
  • Kim, Taejin
  • Youn, Byeng D.

Abstract

Computer-aided engineering (CAE) models have been indispensable to virtual testing for designing and evaluating engineered systems to satisfy reliability requirements. However, it is not easy to fully characterize the variability in the model input variables due to limited resources. Statistical model calibration is thus of great importance as a strategy to improve the predictive capability of a CAE model. Optimization-based statistical model calibration is formulated as an unconstrained optimization problem that infers the unknown statistical parameters of input variables associated with a CAE model by maximizing statistical similarity between predicted and observed output responses. A calibration metric is defined as the objective function to be maximized that quantifies statistical similarity. One important challenge in formulating a calibration metric is how to properly consider the statistical correlation in output responses. Thus, this study proposes a new calibration metric: The Marginal Probability and Correlation Residual (MPCR). The foundational idea of the MPCR is to decompose a multivariate joint probability distribution into multiple marginal probability distributions, while considering the statistical correlation between output responses. The MPCR has favorable properties, such as normalization, boundedness, and marginalization. Two mathematical and two engineering examples are presented to demonstrate the effectiveness and potential benefits of the MPCR.

Suggested Citation

  • Kim, Wongon & Yoon, Heonjun & Lee, Guesuk & Kim, Taejin & Youn, Byeng D., 2020. "A new calibration metric that considers statistical correlation: Marginal Probability and Correlation Residuals," Reliability Engineering and System Safety, Elsevier, vol. 195(C).
    • Handle: RePEc:eee:reensy:v:195:y:2020:i:c:s0951832019301334
    DOI: 10.1016/j.ress.2019.106677
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0951832019301334
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ress.2019.106677?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Zhao, Lufeng & Lu, Zhenzhou & Yun, Wanying & Wang, Wenjin, 2017. "Validation metric based on Mahalanobis distance for models with multiple correlated responses," Reliability Engineering and System Safety, Elsevier, vol. 159(C), pages 80-89.
    2. Li, Chenzhao & Mahadevan, Sankaran, 2016. "Role of calibration, validation, and relevance in multi-level uncertainty integration," Reliability Engineering and System Safety, Elsevier, vol. 148(C), pages 32-43.
    3. Li, Wei & Chen, Wei & Jiang, Zhen & Lu, Zhenzhou & Liu, Yu, 2014. "New validation metrics for models with multiple correlated responses," Reliability Engineering and System Safety, Elsevier, vol. 127(C), pages 1-11.
    4. Kanso, A. & Chebbo, G. & Tassin, B., 2006. "Application of MCMC–GSA model calibration method to urban runoff quality modeling," Reliability Engineering and System Safety, Elsevier, vol. 91(10), pages 1398-1405.
    5. Higdon, Dave & Gattiker, James & Williams, Brian & Rightley, Maria, 2008. "Computer Model Calibration Using High-Dimensional Output," Journal of the American Statistical Association, American Statistical Association, vol. 103, pages 570-583, June.
    6. Cheung, Sai Hung & Oliver, Todd A. & Prudencio, Ernesto E. & Prudhomme, Serge & Moser, Robert D., 2011. "Bayesian uncertainty analysis with applications to turbulence modeling," Reliability Engineering and System Safety, Elsevier, vol. 96(9), pages 1137-1149.
    7. Wang, Chong & Matthies, Hermann G., 2019. "Novel model calibration method via non-probabilistic interval characterization and Bayesian theory," Reliability Engineering and System Safety, Elsevier, vol. 183(C), pages 84-92.
    8. Campbell, Katherine, 2006. "Statistical calibration of computer simulations," Reliability Engineering and System Safety, Elsevier, vol. 91(10), pages 1358-1363.
    9. Marc C. Kennedy & Anthony O'Hagan, 2001. "Bayesian calibration of computer models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 63(3), pages 425-464.
    10. Nagler, Thomas & Czado, Claudia, 2016. "Evading the curse of dimensionality in nonparametric density estimation with simplified vine copulas," Journal of Multivariate Analysis, Elsevier, vol. 151(C), pages 69-89.
    11. Trucano, T.G. & Swiler, L.P. & Igusa, T. & Oberkampf, W.L. & Pilch, M., 2006. "Calibration, validation, and sensitivity analysis: What's what," Reliability Engineering and System Safety, Elsevier, vol. 91(10), pages 1331-1357.
    12. Duong, Tarn, 2007. "ks: Kernel Density Estimation and Kernel Discriminant Analysis for Multivariate Data in R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 21(i07).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jung, Yongsu & Lee, Ikjin, 2021. "Optimal design of experiments for optimization-based model calibration using Fisher information matrix," Reliability Engineering and System Safety, Elsevier, vol. 216(C).
    2. Yoo, Yeongmin & Jung, Ui-Jin & Han, Yong Ha & Lee, Jongsoo, 2021. "Data Augmentation-Based Prediction of System Level Performance under Model and Parameter Uncertainties: Role of Designable Generative Adversarial Networks (DGAN)," Reliability Engineering and System Safety, Elsevier, vol. 206(C).
    3. Kim, Wongon & Lee, Guesuk & Son, Hyejeong & Choi, Hyunhee & Youn, Byeng D., 2022. "Estimation of fatigue crack initiation and growth in engineering product development using a digital twin approach," Reliability Engineering and System Safety, Elsevier, vol. 226(C).
    4. ZHAI, Cheng-lin & CHEN, Xiao-wei, 2020. "Probability damage calculation of building targets under the missile warhead strike," Reliability Engineering and System Safety, Elsevier, vol. 202(C).
    5. Jung, Yongsu & Jo, Hwisang & Choo, Jeonghwan & Lee, Ikjin, 2022. "Statistical model calibration and design optimization under aleatory and epistemic uncertainty," Reliability Engineering and System Safety, Elsevier, vol. 222(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Vanslette, Kevin & Tohme, Tony & Youcef-Toumi, Kamal, 2020. "A general model validation and testing tool," Reliability Engineering and System Safety, Elsevier, vol. 195(C).
    2. Jung, Yongsu & Jo, Hwisang & Choo, Jeonghwan & Lee, Ikjin, 2022. "Statistical model calibration and design optimization under aleatory and epistemic uncertainty," Reliability Engineering and System Safety, Elsevier, vol. 222(C).
    3. Drignei, Dorin, 2011. "A general statistical model for computer experiments with time series output," Reliability Engineering and System Safety, Elsevier, vol. 96(4), pages 460-467.
    4. Yuan, Jun & Ng, Szu Hui, 2013. "A sequential approach for stochastic computer model calibration and prediction," Reliability Engineering and System Safety, Elsevier, vol. 111(C), pages 273-286.
    5. Perrin, G., 2020. "Adaptive calibration of a computer code with time-series output," Reliability Engineering and System Safety, Elsevier, vol. 196(C).
    6. Wu, Xu & Kozlowski, Tomasz & Meidani, Hadi, 2018. "Kriging-based inverse uncertainty quantification of nuclear fuel performance code BISON fission gas release model using time series measurement data," Reliability Engineering and System Safety, Elsevier, vol. 169(C), pages 422-436.
    7. Wilkinson Richard David, 2013. "Approximate Bayesian computation (ABC) gives exact results under the assumption of model error," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 12(2), pages 129-141, May.
    8. Jung, Yongsu & Lee, Ikjin, 2021. "Optimal design of experiments for optimization-based model calibration using Fisher information matrix," Reliability Engineering and System Safety, Elsevier, vol. 216(C).
    9. Park, Inseok & Grandhi, Ramana V., 2014. "A Bayesian statistical method for quantifying model form uncertainty and two model combination methods," Reliability Engineering and System Safety, Elsevier, vol. 129(C), pages 46-56.
    10. White, Staci A. & Herbei, Radu, 2015. "A Monte Carlo approach to quantifying model error in Bayesian parameter estimation," Computational Statistics & Data Analysis, Elsevier, vol. 83(C), pages 168-181.
    11. Merle, X. & Cinnella, P., 2019. "Robust prediction of dense gas flows under uncertain thermodynamic models," Reliability Engineering and System Safety, Elsevier, vol. 183(C), pages 400-421.
    12. Zhang, Jincheng & Zhao, Xiaowei, 2020. "Quantification of parameter uncertainty in wind farm wake modeling," Energy, Elsevier, vol. 196(C).
    13. Paulo, Rui & García-Donato, Gonzalo & Palomo, Jesús, 2012. "Calibration of computer models with multivariate output," Computational Statistics & Data Analysis, Elsevier, vol. 56(12), pages 3959-3974.
    14. Matthew Plumlee, 2014. "Fast Prediction of Deterministic Functions Using Sparse Grid Experimental Designs," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(508), pages 1581-1591, December.
    15. Daniel W. Gladish & Daniel E. Pagendam & Luk J. M. Peeters & Petra M. Kuhnert & Jai Vaze, 2018. "Emulation Engines: Choice and Quantification of Uncertainty for Complex Hydrological Models," Journal of Agricultural, Biological and Environmental Statistics, Springer;The International Biometric Society;American Statistical Association, vol. 23(1), pages 39-62, March.
    16. Chen, Yewen & Chang, Xiaohui & Luo, Fangzhi & Huang, Hui, 2023. "Additive dynamic models for correcting numerical model outputs," Computational Statistics & Data Analysis, Elsevier, vol. 187(C).
    17. Li, Luyi & Lu, Zhenzhou, 2018. "A new method for model validation with multivariate output," Reliability Engineering and System Safety, Elsevier, vol. 169(C), pages 579-592.
    18. Bruno Sansó & Chris Forest, 2009. "Statistical calibration of climate system properties," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 58(4), pages 485-503, September.
    19. Wu, Danqing & Lu, Zhenzhou & Wang, Yanping & Cheng, Lei, 2015. "Model validation and calibration based on component functions of model output," Reliability Engineering and System Safety, Elsevier, vol. 140(C), pages 59-70.
    20. Antony M. Overstall & David C. Woods, 2013. "A Strategy for Bayesian Inference for Computationally Expensive Models with Application to the Estimation of Stem Cell Properties," Biometrics, The International Biometric Society, vol. 69(2), pages 458-468, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:reensy:v:195:y:2020:i:c:s0951832019301334. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/reliability-engineering-and-system-safety .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.