IDEAS home Printed from https://ideas.repec.org/a/eee/reensy/v96y2011i9p1208-1219.html
   My bibliography  Save this article

Batch sequential design to achieve predictive maturity with calibrated computer models

Author

Listed:
  • Williams, Brian J.
  • Loeppky, Jason L.
  • Moore, Leslie M.
  • Macklem, Mason S.

Abstract

Sequential experiment design strategies have been proposed for efficiently augmenting initial designs to solve many problems of interest to computer experimenters, including optimization, contour and threshold estimation, and global prediction. We focus on batch sequential design strategies for achieving maturity in global prediction of discrepancy inferred from computer model calibration. Predictive maturity focuses on adding field experiments to efficiently improve discrepancy inference. Several design criteria are extended to allow batch augmentation, including integrated and maximum mean square error, maximum entropy, and two expected improvement criteria. In addition, batch versions of maximin distance and weighted distance criteria are developed. Two batch optimization algorithms are considered: modified Fedorov exchange and a binning methodology motivated by optimizing augmented fractional factorial skeleton designs.

Suggested Citation

  • Williams, Brian J. & Loeppky, Jason L. & Moore, Leslie M. & Macklem, Mason S., 2011. "Batch sequential design to achieve predictive maturity with calibrated computer models," Reliability Engineering and System Safety, Elsevier, vol. 96(9), pages 1208-1219.
    • Handle: RePEc:eee:reensy:v:96:y:2011:i:9:p:1208-1219
    DOI: 10.1016/j.ress.2010.04.017
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S095183201100072X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ress.2010.04.017?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Higdon, Dave & Gattiker, James & Williams, Brian & Rightley, Maria, 2008. "Computer Model Calibration Using High-Dimensional Output," Journal of the American Statistical Association, American Statistical Association, vol. 103, pages 570-583, June.
    2. Rommel Regis & Christine Shoemaker, 2005. "Constrained Global Optimization of Expensive Black Box Functions Using Radial Basis Functions," Journal of Global Optimization, Springer, vol. 31(1), pages 153-171, January.
    3. Marc C. Kennedy & Anthony O'Hagan, 2001. "Bayesian calibration of computer models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 63(3), pages 425-464.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Leatherman, Erin R. & Dean, Angela M. & Santner, Thomas J., 2017. "Designing combined physical and computer experiments to maximize prediction accuracy," Computational Statistics & Data Analysis, Elsevier, vol. 113(C), pages 346-362.
    2. Yuan, Jun & Ng, Szu Hui, 2013. "A sequential approach for stochastic computer model calibration and prediction," Reliability Engineering and System Safety, Elsevier, vol. 111(C), pages 273-286.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Drignei, Dorin, 2011. "A general statistical model for computer experiments with time series output," Reliability Engineering and System Safety, Elsevier, vol. 96(4), pages 460-467.
    2. Hwang, Youngdeok & Kim, Hang J. & Chang, Won & Yeo, Kyongmin & Kim, Yongku, 2019. "Bayesian pollution source identification via an inverse physics model," Computational Statistics & Data Analysis, Elsevier, vol. 134(C), pages 76-92.
    3. Ioannis Andrianakis & Ian R Vernon & Nicky McCreesh & Trevelyan J McKinley & Jeremy E Oakley & Rebecca N Nsubuga & Michael Goldstein & Richard G White, 2015. "Bayesian History Matching of Complex Infectious Disease Models Using Emulation: A Tutorial and a Case Study on HIV in Uganda," PLOS Computational Biology, Public Library of Science, vol. 11(1), pages 1-18, January.
    4. Samantha M. Roth & Ben Seiyon Lee & Sanjib Sharma & Iman Hosseini‐Shakib & Klaus Keller & Murali Haran, 2023. "Flood hazard model calibration using multiresolution model output," Environmetrics, John Wiley & Sons, Ltd., vol. 34(2), March.
    5. Perrin, G., 2020. "Adaptive calibration of a computer code with time-series output," Reliability Engineering and System Safety, Elsevier, vol. 196(C).
    6. White, Staci A. & Herbei, Radu, 2015. "A Monte Carlo approach to quantifying model error in Bayesian parameter estimation," Computational Statistics & Data Analysis, Elsevier, vol. 83(C), pages 168-181.
    7. Mevin Hooten & Christopher Wikle & Michael Schwob, 2020. "Statistical Implementations of Agent‐Based Demographic Models," International Statistical Review, International Statistical Institute, vol. 88(2), pages 441-461, August.
    8. Leatherman, Erin R. & Dean, Angela M. & Santner, Thomas J., 2017. "Designing combined physical and computer experiments to maximize prediction accuracy," Computational Statistics & Data Analysis, Elsevier, vol. 113(C), pages 346-362.
    9. Wu, Xu & Kozlowski, Tomasz & Meidani, Hadi, 2018. "Kriging-based inverse uncertainty quantification of nuclear fuel performance code BISON fission gas release model using time series measurement data," Reliability Engineering and System Safety, Elsevier, vol. 169(C), pages 422-436.
    10. Sun, Yang & Fang, Xiangzhong, 2024. "Efficient calibration of computer models with multivariate output," Journal of Multivariate Analysis, Elsevier, vol. 202(C).
    11. Nott, David J. & Marshall, Lucy & Fielding, Mark & Liong, Shie-Yui, 2014. "Mixtures of experts for understanding model discrepancy in dynamic computer models," Computational Statistics & Data Analysis, Elsevier, vol. 71(C), pages 491-505.
    12. Paulo, Rui & García-Donato, Gonzalo & Palomo, Jesús, 2012. "Calibration of computer models with multivariate output," Computational Statistics & Data Analysis, Elsevier, vol. 56(12), pages 3959-3974.
    13. Jackson Samuel E. & Vernon Ian & Liu Junli & Lindsey Keith, 2020. "Understanding hormonal crosstalk in Arabidopsis root development via emulation and history matching," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 19(2), pages 1-33, April.
    14. Guillaume Perrin & Christian Soize, 2020. "Adaptive method for indirect identification of the statistical properties of random fields in a Bayesian framework," Computational Statistics, Springer, vol. 35(1), pages 111-133, March.
    15. Stripling, H.F. & Adams, M.L. & McClarren, R.G. & Mallick, B.K., 2011. "The Method of Manufactured Universes for validating uncertainty quantification methods," Reliability Engineering and System Safety, Elsevier, vol. 96(9), pages 1242-1256.
    16. Matthew Plumlee, 2014. "Fast Prediction of Deterministic Functions Using Sparse Grid Experimental Designs," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(508), pages 1581-1591, December.
    17. SungKu Kang & Ran Jin & Xinwei Deng & Ron S. Kenett, 2023. "Challenges of modeling and analysis in cybermanufacturing: a review from a machine learning and computation perspective," Journal of Intelligent Manufacturing, Springer, vol. 34(2), pages 415-428, February.
    18. Manfren, Massimiliano & Aste, Niccolò & Moshksar, Reza, 2013. "Calibration and uncertainty analysis for computer models – A meta-model based approach for integrated building energy simulation," Applied Energy, Elsevier, vol. 103(C), pages 627-641.
    19. Daniel W. Gladish & Daniel E. Pagendam & Luk J. M. Peeters & Petra M. Kuhnert & Jai Vaze, 2018. "Emulation Engines: Choice and Quantification of Uncertainty for Complex Hydrological Models," Journal of Agricultural, Biological and Environmental Statistics, Springer;The International Biometric Society;American Statistical Association, vol. 23(1), pages 39-62, March.
    20. Chen, Yewen & Chang, Xiaohui & Luo, Fangzhi & Huang, Hui, 2023. "Additive dynamic models for correcting numerical model outputs," Computational Statistics & Data Analysis, Elsevier, vol. 187(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:reensy:v:96:y:2011:i:9:p:1208-1219. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/reliability-engineering-and-system-safety .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.