IDEAS home Printed from https://ideas.repec.org/a/eee/reensy/v96y2011i9p1208-1219.html
   My bibliography  Save this article

Batch sequential design to achieve predictive maturity with calibrated computer models

Author

Listed:
  • Williams, Brian J.
  • Loeppky, Jason L.
  • Moore, Leslie M.
  • Macklem, Mason S.

Abstract

Sequential experiment design strategies have been proposed for efficiently augmenting initial designs to solve many problems of interest to computer experimenters, including optimization, contour and threshold estimation, and global prediction. We focus on batch sequential design strategies for achieving maturity in global prediction of discrepancy inferred from computer model calibration. Predictive maturity focuses on adding field experiments to efficiently improve discrepancy inference. Several design criteria are extended to allow batch augmentation, including integrated and maximum mean square error, maximum entropy, and two expected improvement criteria. In addition, batch versions of maximin distance and weighted distance criteria are developed. Two batch optimization algorithms are considered: modified Fedorov exchange and a binning methodology motivated by optimizing augmented fractional factorial skeleton designs.

Suggested Citation

  • Williams, Brian J. & Loeppky, Jason L. & Moore, Leslie M. & Macklem, Mason S., 2011. "Batch sequential design to achieve predictive maturity with calibrated computer models," Reliability Engineering and System Safety, Elsevier, vol. 96(9), pages 1208-1219.
    • Handle: RePEc:eee:reensy:v:96:y:2011:i:9:p:1208-1219
    DOI: 10.1016/j.ress.2010.04.017
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S095183201100072X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ress.2010.04.017?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Rommel Regis & Christine Shoemaker, 2005. "Constrained Global Optimization of Expensive Black Box Functions Using Radial Basis Functions," Journal of Global Optimization, Springer, vol. 31(1), pages 153-171, January.
    2. Higdon, Dave & Gattiker, James & Williams, Brian & Rightley, Maria, 2008. "Computer Model Calibration Using High-Dimensional Output," Journal of the American Statistical Association, American Statistical Association, vol. 103, pages 570-583, June.
    3. Marc C. Kennedy & Anthony O'Hagan, 2001. "Bayesian calibration of computer models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 63(3), pages 425-464.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Leatherman, Erin R. & Dean, Angela M. & Santner, Thomas J., 2017. "Designing combined physical and computer experiments to maximize prediction accuracy," Computational Statistics & Data Analysis, Elsevier, vol. 113(C), pages 346-362.
    2. Yuan, Jun & Ng, Szu Hui, 2013. "A sequential approach for stochastic computer model calibration and prediction," Reliability Engineering and System Safety, Elsevier, vol. 111(C), pages 273-286.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Drignei, Dorin, 2011. "A general statistical model for computer experiments with time series output," Reliability Engineering and System Safety, Elsevier, vol. 96(4), pages 460-467.
    2. White, Staci A. & Herbei, Radu, 2015. "A Monte Carlo approach to quantifying model error in Bayesian parameter estimation," Computational Statistics & Data Analysis, Elsevier, vol. 83(C), pages 168-181.
    3. Wu, Xu & Kozlowski, Tomasz & Meidani, Hadi, 2018. "Kriging-based inverse uncertainty quantification of nuclear fuel performance code BISON fission gas release model using time series measurement data," Reliability Engineering and System Safety, Elsevier, vol. 169(C), pages 422-436.
    4. Sun, Yang & Fang, Xiangzhong, 2024. "Efficient calibration of computer models with multivariate output," Journal of Multivariate Analysis, Elsevier, vol. 202(C).
    5. Paulo, Rui & García-Donato, Gonzalo & Palomo, Jesús, 2012. "Calibration of computer models with multivariate output," Computational Statistics & Data Analysis, Elsevier, vol. 56(12), pages 3959-3974.
    6. Matthew Plumlee, 2014. "Fast Prediction of Deterministic Functions Using Sparse Grid Experimental Designs," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(508), pages 1581-1591, December.
    7. Daniel W. Gladish & Daniel E. Pagendam & Luk J. M. Peeters & Petra M. Kuhnert & Jai Vaze, 2018. "Emulation Engines: Choice and Quantification of Uncertainty for Complex Hydrological Models," Journal of Agricultural, Biological and Environmental Statistics, Springer;The International Biometric Society;American Statistical Association, vol. 23(1), pages 39-62, March.
    8. Chen, Yewen & Chang, Xiaohui & Luo, Fangzhi & Huang, Hui, 2023. "Additive dynamic models for correcting numerical model outputs," Computational Statistics & Data Analysis, Elsevier, vol. 187(C).
    9. Bruno Sansó & Chris Forest, 2009. "Statistical calibration of climate system properties," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 58(4), pages 485-503, September.
    10. Wilkinson Richard David, 2013. "Approximate Bayesian computation (ABC) gives exact results under the assumption of model error," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 12(2), pages 129-141, May.
    11. Antony M. Overstall & David C. Woods, 2013. "A Strategy for Bayesian Inference for Computationally Expensive Models with Application to the Estimation of Stem Cell Properties," Biometrics, The International Biometric Society, vol. 69(2), pages 458-468, June.
    12. Won Chang & Murali Haran & Patrick Applegate & David Pollard, 2016. "Calibrating an Ice Sheet Model Using High-Dimensional Binary Spatial Data," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(513), pages 57-72, March.
    13. Mohammadi, Hossein & Challenor, Peter & Goodfellow, Marc, 2019. "Emulating dynamic non-linear simulators using Gaussian processes," Computational Statistics & Data Analysis, Elsevier, vol. 139(C), pages 178-196.
    14. Luca Aiello & Matteo Fontana & Alessandra Guglielmi, 2023. "Bayesian functional emulation of CO2 emissions on future climate change scenarios," Environmetrics, John Wiley & Sons, Ltd., vol. 34(8), December.
    15. Jung, Yongsu & Lee, Ikjin, 2021. "Optimal design of experiments for optimization-based model calibration using Fisher information matrix," Reliability Engineering and System Safety, Elsevier, vol. 216(C).
    16. Kim, Wongon & Lee, Guesuk & Son, Hyejeong & Choi, Hyunhee & Youn, Byeng D., 2022. "Estimation of fatigue crack initiation and growth in engineering product development using a digital twin approach," Reliability Engineering and System Safety, Elsevier, vol. 226(C).
    17. Giri Gopalan & Christopher K. Wikle, 2022. "A Higher-Order Singular Value Decomposition Tensor Emulator for Spatiotemporal Simulators," Journal of Agricultural, Biological and Environmental Statistics, Springer;The International Biometric Society;American Statistical Association, vol. 27(1), pages 22-45, March.
    18. Maupin, Kathryn A. & Swiler, Laura P., 2020. "Model discrepancy calibration across experimental settings," Reliability Engineering and System Safety, Elsevier, vol. 200(C).
    19. Curtis B. Storlie & William A. Lane & Emily M. Ryan & James R. Gattiker & David M. Higdon, 2015. "Calibration of Computational Models With Categorical Parameters and Correlated Outputs via Bayesian Smoothing Spline ANOVA," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(509), pages 68-82, March.
    20. Hwang, Youngdeok & Kim, Hang J. & Chang, Won & Yeo, Kyongmin & Kim, Yongku, 2019. "Bayesian pollution source identification via an inverse physics model," Computational Statistics & Data Analysis, Elsevier, vol. 134(C), pages 76-92.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:reensy:v:96:y:2011:i:9:p:1208-1219. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/reliability-engineering-and-system-safety .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.