IDEAS home Printed from https://ideas.repec.org/a/eee/reensy/v216y2021ics0951832021004798.html
   My bibliography  Save this article

Optimal design of experiments for optimization-based model calibration using Fisher information matrix

Author

Listed:
  • Jung, Yongsu
  • Lee, Ikjin

Abstract

Statistical model calibration to infer unknown model parameters and model bias has been widely developed through comparison between simulation response and experimental data. Bayesian-based model calibration typically represented as Kennedy and O'Hagan (KOH) framework and optimization-based model calibration have been proposed, but efforts on optimization of experimental design to reduce the epistemic uncertainty minimizing experimental resources are still limited. Furthermore, when an unknown model parameter has natural and uncontrollable variability, the estimation may be much more difficult since both aleatory and epistemic uncertainties exist. In this paper, we have developed a framework to find the optimal design of experiments (DoE) satisfying the target information gain for inference of unknown model parameters based on optimization-based model calibration. The expected Fisher information matrix is approximated to quantify the expected information gain for the maximum likelihood estimation (MLE). Namely, the necessary number of experiments at each experimental design can be obtained to attain desired precision on estimators while minimizing the overall experimental cost. The numerical study verifies the feasibility of the proposed framework, which means that there is certainly a dominant DoE that gives more information to the inference on specific model parameters.

Suggested Citation

  • Jung, Yongsu & Lee, Ikjin, 2021. "Optimal design of experiments for optimization-based model calibration using Fisher information matrix," Reliability Engineering and System Safety, Elsevier, vol. 216(C).
  • Handle: RePEc:eee:reensy:v:216:y:2021:i:c:s0951832021004798
    DOI: 10.1016/j.ress.2021.107968
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0951832021004798
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.ress.2021.107968?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Hou, Tianfeng & Nuyens, Dirk & Roels, Staf & Janssen, Hans, 2019. "Quasi-Monte Carlo based uncertainty analysis: Sampling efficiency and error estimation in engineering applications," Reliability Engineering and System Safety, Elsevier, vol. 191(C).
    2. Elizabeth G. Ryan & Christopher C. Drovandi & James M. McGree & Anthony N. Pettitt, 2016. "A Review of Modern Computational Algorithms for Bayesian Optimal Design," International Statistical Review, International Statistical Institute, vol. 84(1), pages 128-154, April.
    3. Nguyen, Son & Chen, Peggy Shu-Ling & Du, Yuquan & Thai, Vinh V., 2021. "An Operational Risk Analysis Model for Container Shipping Systems considering Uncertainty Quantification," Reliability Engineering and System Safety, Elsevier, vol. 209(C).
    4. Sankararaman, Shankar & Mahadevan, Sankaran, 2015. "Integration of model verification, validation, and calibration for uncertainty quantification in engineering systems," Reliability Engineering and System Safety, Elsevier, vol. 138(C), pages 194-209.
    5. Liu, Yushan & Li, Luyi & Zhao, Sihan & Song, Shufang, 2021. "A global surrogate model technique based on principal component analysis and Kriging for uncertainty propagation of dynamic systems," Reliability Engineering and System Safety, Elsevier, vol. 207(C).
    6. Higdon, Dave & Gattiker, James & Williams, Brian & Rightley, Maria, 2008. "Computer Model Calibration Using High-Dimensional Output," Journal of the American Statistical Association, American Statistical Association, vol. 103, pages 570-583, June.
    7. Kim, Wongon & Yoon, Heonjun & Lee, Guesuk & Kim, Taejin & Youn, Byeng D., 2020. "A new calibration metric that considers statistical correlation: Marginal Probability and Correlation Residuals," Reliability Engineering and System Safety, Elsevier, vol. 195(C).
    8. Paul D. Arendt & Daniel W. Apley & Wei Chen, 2016. "A preposterior analysis to predict identifiability in the experimental calibration of computer models," IISE Transactions, Taylor & Francis Journals, vol. 48(1), pages 75-88, January.
    9. Ao, Dan & Hu, Zhen & Mahadevan, Sankaran, 2017. "Design of validation experiments for life prediction models," Reliability Engineering and System Safety, Elsevier, vol. 165(C), pages 22-33.
    10. Marc C. Kennedy & Anthony O'Hagan, 2001. "Bayesian calibration of computer models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 63(3), pages 425-464.
    11. Wu, Shuo-Jye & Hsu, Chu-Chun & Huang, Syuan-Rong, 2020. "Optimal designs and reliability sampling plans for one-shot devices with cost considerations," Reliability Engineering and System Safety, Elsevier, vol. 197(C).
    12. Li, Chenzhao & Mahadevan, Sankaran, 2016. "Role of calibration, validation, and relevance in multi-level uncertainty integration," Reliability Engineering and System Safety, Elsevier, vol. 148(C), pages 32-43.
    13. Paul Larsen, 2015. "Asyptotic Normality for Maximum Likelihood Estimation and Operational Risk," Papers 1508.02824, arXiv.org, revised Aug 2016.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yao Tong & Duo Zhang & Zhijiang Shao & Xiaojin Huang, 2023. "Global Model Calibration of High-Temperature Gas-Cooled Reactor Pebble-Bed Module Using an Adaptive Experimental Design," Energies, MDPI, vol. 16(12), pages 1-25, June.
    2. Alimohammadi, H.R. & Naseh, H. & Ommi, F., 2023. "An integrated methodology applied for reliability based multi-disciplinary design optimization in EPFE with LOX/kerosene," Reliability Engineering and System Safety, Elsevier, vol. 240(C).
    3. Zheng, Huiling & Yang, Jun & Xu, Houbao & Zhao, Yu, 2023. "Reliability acceptance sampling plan for degraded products subject to Wiener process with unit heterogeneity," Reliability Engineering and System Safety, Elsevier, vol. 229(C).
    4. Jung, Yongsu & Jo, Hwisang & Choo, Jeonghwan & Lee, Ikjin, 2022. "Statistical model calibration and design optimization under aleatory and epistemic uncertainty," Reliability Engineering and System Safety, Elsevier, vol. 222(C).
    5. Zhu, Xiaojun & Balakrishnan, N., 2022. "One-shot device test data analysis using non-parametric and semi-parametric inferential methods and applications," Reliability Engineering and System Safety, Elsevier, vol. 221(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jung, Yongsu & Jo, Hwisang & Choo, Jeonghwan & Lee, Ikjin, 2022. "Statistical model calibration and design optimization under aleatory and epistemic uncertainty," Reliability Engineering and System Safety, Elsevier, vol. 222(C).
    2. Vanslette, Kevin & Tohme, Tony & Youcef-Toumi, Kamal, 2020. "A general model validation and testing tool," Reliability Engineering and System Safety, Elsevier, vol. 195(C).
    3. Perrin, G., 2020. "Adaptive calibration of a computer code with time-series output," Reliability Engineering and System Safety, Elsevier, vol. 196(C).
    4. Kim, Wongon & Yoon, Heonjun & Lee, Guesuk & Kim, Taejin & Youn, Byeng D., 2020. "A new calibration metric that considers statistical correlation: Marginal Probability and Correlation Residuals," Reliability Engineering and System Safety, Elsevier, vol. 195(C).
    5. Yoo, Yeongmin & Jung, Ui-Jin & Han, Yong Ha & Lee, Jongsoo, 2021. "Data Augmentation-Based Prediction of System Level Performance under Model and Parameter Uncertainties: Role of Designable Generative Adversarial Networks (DGAN)," Reliability Engineering and System Safety, Elsevier, vol. 206(C).
    6. Kim, Wongon & Lee, Guesuk & Son, Hyejeong & Choi, Hyunhee & Youn, Byeng D., 2022. "Estimation of fatigue crack initiation and growth in engineering product development using a digital twin approach," Reliability Engineering and System Safety, Elsevier, vol. 226(C).
    7. Maupin, Kathryn A. & Swiler, Laura P., 2020. "Model discrepancy calibration across experimental settings," Reliability Engineering and System Safety, Elsevier, vol. 200(C).
    8. Drignei, Dorin, 2011. "A general statistical model for computer experiments with time series output," Reliability Engineering and System Safety, Elsevier, vol. 96(4), pages 460-467.
    9. Hwang, Youngdeok & Kim, Hang J. & Chang, Won & Yeo, Kyongmin & Kim, Yongku, 2019. "Bayesian pollution source identification via an inverse physics model," Computational Statistics & Data Analysis, Elsevier, vol. 134(C), pages 76-92.
    10. Ioannis Andrianakis & Ian R Vernon & Nicky McCreesh & Trevelyan J McKinley & Jeremy E Oakley & Rebecca N Nsubuga & Michael Goldstein & Richard G White, 2015. "Bayesian History Matching of Complex Infectious Disease Models Using Emulation: A Tutorial and a Case Study on HIV in Uganda," PLOS Computational Biology, Public Library of Science, vol. 11(1), pages 1-18, January.
    11. Samantha M. Roth & Ben Seiyon Lee & Sanjib Sharma & Iman Hosseini‐Shakib & Klaus Keller & Murali Haran, 2023. "Flood hazard model calibration using multiresolution model output," Environmetrics, John Wiley & Sons, Ltd., vol. 34(2), March.
    12. Zhang, Yang & Xu, Jun & Gardoni, Paolo, 2024. "A loading contribution degree analysis-based strategy for time-variant reliability analysis of structures under multiple loading stochastic processes," Reliability Engineering and System Safety, Elsevier, vol. 243(C).
    13. White, Staci A. & Herbei, Radu, 2015. "A Monte Carlo approach to quantifying model error in Bayesian parameter estimation," Computational Statistics & Data Analysis, Elsevier, vol. 83(C), pages 168-181.
    14. Mevin Hooten & Christopher Wikle & Michael Schwob, 2020. "Statistical Implementations of Agent‐Based Demographic Models," International Statistical Review, International Statistical Institute, vol. 88(2), pages 441-461, August.
    15. Leatherman, Erin R. & Dean, Angela M. & Santner, Thomas J., 2017. "Designing combined physical and computer experiments to maximize prediction accuracy," Computational Statistics & Data Analysis, Elsevier, vol. 113(C), pages 346-362.
    16. Wu, Xu & Kozlowski, Tomasz & Meidani, Hadi, 2018. "Kriging-based inverse uncertainty quantification of nuclear fuel performance code BISON fission gas release model using time series measurement data," Reliability Engineering and System Safety, Elsevier, vol. 169(C), pages 422-436.
    17. Sun, Yang & Fang, Xiangzhong, 2024. "Efficient calibration of computer models with multivariate output," Journal of Multivariate Analysis, Elsevier, vol. 202(C).
    18. Joel A Paulson & Marc Martin-Casas & Ali Mesbah, 2019. "Fast uncertainty quantification for dynamic flux balance analysis using non-smooth polynomial chaos expansions," PLOS Computational Biology, Public Library of Science, vol. 15(8), pages 1-35, August.
    19. Nott, David J. & Marshall, Lucy & Fielding, Mark & Liong, Shie-Yui, 2014. "Mixtures of experts for understanding model discrepancy in dynamic computer models," Computational Statistics & Data Analysis, Elsevier, vol. 71(C), pages 491-505.
    20. Paulo, Rui & García-Donato, Gonzalo & Palomo, Jesús, 2012. "Calibration of computer models with multivariate output," Computational Statistics & Data Analysis, Elsevier, vol. 56(12), pages 3959-3974.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:reensy:v:216:y:2021:i:c:s0951832021004798. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/reliability-engineering-and-system-safety .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.