IDEAS home Printed from https://ideas.repec.org/a/eee/reensy/v140y2015icp59-70.html
   My bibliography  Save this article

Model validation and calibration based on component functions of model output

Author

Listed:
  • Wu, Danqing
  • Lu, Zhenzhou
  • Wang, Yanping
  • Cheng, Lei

Abstract

The target in this work is to validate the component functions of model output between physical observation and computational model with the area metric. Based on the theory of high dimensional model representations (HDMR) of independent input variables, conditional expectations are component functions of model output, and the conditional expectations reflect partial information of model output. Therefore, the model validation of conditional expectations tells the discrepancy between the partial information of the computational model output and that of the observations. Then a calibration of the conditional expectations is carried out to reduce the value of model validation metric. After that, a recalculation of the model validation metric of model output is taken with the calibrated model parameters, and the result shows that a reduction of the discrepancy in the conditional expectations can help decrease the difference in model output. At last, several examples are employed to demonstrate the rationality and necessity of the methodology in case of both single validation site and multiple validation sites.

Suggested Citation

  • Wu, Danqing & Lu, Zhenzhou & Wang, Yanping & Cheng, Lei, 2015. "Model validation and calibration based on component functions of model output," Reliability Engineering and System Safety, Elsevier, vol. 140(C), pages 59-70.
  • Handle: RePEc:eee:reensy:v:140:y:2015:i:c:p:59-70
    DOI: 10.1016/j.ress.2015.03.024
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0951832015000903
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.ress.2015.03.024?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Scott Ferson & William L. Oberkampf, 2009. "Validation of imprecise probability models," International Journal of Reliability and Safety, Inderscience Enterprises Ltd, vol. 3(1/2/3), pages 3-22.
    2. Marc C. Kennedy & Anthony O'Hagan, 2001. "Bayesian calibration of computer models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 63(3), pages 425-464.
    3. Trucano, T.G. & Swiler, L.P. & Igusa, T. & Oberkampf, W.L. & Pilch, M., 2006. "Calibration, validation, and sensitivity analysis: What's what," Reliability Engineering and System Safety, Elsevier, vol. 91(10), pages 1331-1357.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Vanslette, Kevin & Tohme, Tony & Youcef-Toumi, Kamal, 2020. "A general model validation and testing tool," Reliability Engineering and System Safety, Elsevier, vol. 195(C).
    2. Wang, Chong & Matthies, Hermann G., 2019. "Novel model calibration method via non-probabilistic interval characterization and Bayesian theory," Reliability Engineering and System Safety, Elsevier, vol. 183(C), pages 84-92.
    3. Tohme, Tony & Vanslette, Kevin & Youcef-Toumi, Kamal, 2020. "A generalized Bayesian approach to model calibration," Reliability Engineering and System Safety, Elsevier, vol. 204(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Drignei, Dorin, 2011. "A general statistical model for computer experiments with time series output," Reliability Engineering and System Safety, Elsevier, vol. 96(4), pages 460-467.
    2. Yuan, Jun & Ng, Szu Hui, 2013. "A sequential approach for stochastic computer model calibration and prediction," Reliability Engineering and System Safety, Elsevier, vol. 111(C), pages 273-286.
    3. Wang, Chong & Matthies, Hermann G., 2019. "Novel model calibration method via non-probabilistic interval characterization and Bayesian theory," Reliability Engineering and System Safety, Elsevier, vol. 183(C), pages 84-92.
    4. Kwag, Shinyoung & Gupta, Abhinav & Dinh, Nam, 2018. "Probabilistic risk assessment based model validation method using Bayesian network," Reliability Engineering and System Safety, Elsevier, vol. 169(C), pages 380-393.
    5. Liu, Yang & Wang, Dewei & Sun, Xiaodong & Liu, Yang & Dinh, Nam & Hu, Rui, 2021. "Uncertainty quantification for Multiphase-CFD simulations of bubbly flows: a machine learning-based Bayesian approach supported by high-resolution experiments," Reliability Engineering and System Safety, Elsevier, vol. 212(C).
    6. Zhao, Lufeng & Lu, Zhenzhou & Yun, Wanying & Wang, Wenjin, 2017. "Validation metric based on Mahalanobis distance for models with multiple correlated responses," Reliability Engineering and System Safety, Elsevier, vol. 159(C), pages 80-89.
    7. Vanslette, Kevin & Tohme, Tony & Youcef-Toumi, Kamal, 2020. "A general model validation and testing tool," Reliability Engineering and System Safety, Elsevier, vol. 195(C).
    8. Perrin, G., 2020. "Adaptive calibration of a computer code with time-series output," Reliability Engineering and System Safety, Elsevier, vol. 196(C).
    9. Li, Wei & Chen, Wei & Jiang, Zhen & Lu, Zhenzhou & Liu, Yu, 2014. "New validation metrics for models with multiple correlated responses," Reliability Engineering and System Safety, Elsevier, vol. 127(C), pages 1-11.
    10. Li, Chenzhao & Mahadevan, Sankaran, 2016. "Role of calibration, validation, and relevance in multi-level uncertainty integration," Reliability Engineering and System Safety, Elsevier, vol. 148(C), pages 32-43.
    11. Kim, Wongon & Yoon, Heonjun & Lee, Guesuk & Kim, Taejin & Youn, Byeng D., 2020. "A new calibration metric that considers statistical correlation: Marginal Probability and Correlation Residuals," Reliability Engineering and System Safety, Elsevier, vol. 195(C).
    12. Park, Inseok & Grandhi, Ramana V., 2014. "A Bayesian statistical method for quantifying model form uncertainty and two model combination methods," Reliability Engineering and System Safety, Elsevier, vol. 129(C), pages 46-56.
    13. Mullins, Joshua & Ling, You & Mahadevan, Sankaran & Sun, Lin & Strachan, Alejandro, 2016. "Separation of aleatory and epistemic uncertainty in probabilistic model validation," Reliability Engineering and System Safety, Elsevier, vol. 147(C), pages 49-59.
    14. Matthias Katzfuss & Joseph Guinness & Wenlong Gong & Daniel Zilber, 2020. "Vecchia Approximations of Gaussian-Process Predictions," Journal of Agricultural, Biological and Environmental Statistics, Springer;The International Biometric Society;American Statistical Association, vol. 25(3), pages 383-414, September.
    15. Hao Wu & Michael Browne, 2015. "Random Model Discrepancy: Interpretations and Technicalities (A Rejoinder)," Psychometrika, Springer;The Psychometric Society, vol. 80(3), pages 619-624, September.
    16. Xiaoyu Xiong & Benjamin D. Youngman & Theodoros Economou, 2021. "Data fusion with Gaussian processes for estimation of environmental hazard events," Environmetrics, John Wiley & Sons, Ltd., vol. 32(3), May.
    17. Petropoulos, G. & Wooster, M.J. & Carlson, T.N. & Kennedy, M.C. & Scholze, M., 2009. "A global Bayesian sensitivity analysis of the 1d SimSphere soil–vegetation–atmospheric transfer (SVAT) model using Gaussian model emulation," Ecological Modelling, Elsevier, vol. 220(19), pages 2427-2440.
    18. Helton, Jon C. & Brooks, Dusty M. & Sallaberry, Cédric J., 2020. "Property values associated with the failure of individual links in a system with multiple weak and strong links," Reliability Engineering and System Safety, Elsevier, vol. 195(C).
    19. Edward Boone & Jan Hannig & Ryad Ghanam & Sujit Ghosh & Fabrizio Ruggeri & Serge Prudhomme, 2022. "Model Validation of a Single Degree-of-Freedom Oscillator: A Case Study," Stats, MDPI, vol. 5(4), pages 1-17, November.
    20. Abokersh, Mohamed Hany & Vallès, Manel & Cabeza, Luisa F. & Boer, Dieter, 2020. "A framework for the optimal integration of solar assisted district heating in different urban sized communities: A robust machine learning approach incorporating global sensitivity analysis," Applied Energy, Elsevier, vol. 267(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:reensy:v:140:y:2015:i:c:p:59-70. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/reliability-engineering-and-system-safety .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.