IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v113y2017icp346-362.html
   My bibliography  Save this article

Designing combined physical and computer experiments to maximize prediction accuracy

Author

Listed:
  • Leatherman, Erin R.
  • Dean, Angela M.
  • Santner, Thomas J.

Abstract

Combined designs for experiments involving a physical system and a simulator of the physical system are evaluated in terms of their accuracy of predicting the mean of the physical system. Comparisons are made among designs that are (1) locally optimal under the minimum integrated mean squared prediction error criterion for the combined physical system and simulator experiments, (2) locally optimal for the physical or simulator experiments, with a fixed design for the component not being optimized, (3) maximin augmented nested Latin hypercube, and (4) I-optimal for the physical system experiment and maximin Latin hypercube for the simulator experiment. Computational methods are proposed for constructing the designs of interest. For a large test bed of examples, the empirical mean squared prediction errors are compared at a grid of inputs for each test surface using a statistically calibrated Bayesian predictor based on the data from each design. The prediction errors are also studied for a test bed that varies only the calibration parameter of the test surface. Design recommendations are given.

Suggested Citation

  • Leatherman, Erin R. & Dean, Angela M. & Santner, Thomas J., 2017. "Designing combined physical and computer experiments to maximize prediction accuracy," Computational Statistics & Data Analysis, Elsevier, vol. 113(C), pages 346-362.
    • Handle: RePEc:eee:csdana:v:113:y:2017:i:c:p:346-362
    DOI: 10.1016/j.csda.2016.07.013
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167947316301773
    Download Restriction: Full text for ScienceDirect subscribers only.
    File URL: https://libkey.io/10.1016/j.csda.2016.07.013?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Williams, Brian J. & Loeppky, Jason L. & Moore, Leslie M. & Macklem, Mason S., 2011. "Batch sequential design to achieve predictive maturity with calibrated computer models," Reliability Engineering and System Safety, Elsevier, vol. 96(9), pages 1208-1219.
    2. Higdon, Dave & Gattiker, James & Williams, Brian & Rightley, Maria, 2008. "Computer Model Calibration Using High-Dimensional Output," Journal of the American Statistical Association, American Statistical Association, vol. 103, pages 570-583, June.
    3. Marc C. Kennedy & Anthony O'Hagan, 2001. "Bayesian calibration of computer models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 63(3), pages 425-464.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Huaimin Diao & Yan Wang & Dianpeng Wang, 2022. "A D-Optimal Sequential Calibration Design for Computer Models," Mathematics, MDPI, vol. 10(9), pages 1-15, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Drignei, Dorin, 2011. "A general statistical model for computer experiments with time series output," Reliability Engineering and System Safety, Elsevier, vol. 96(4), pages 460-467.
    2. Yuan, Jun & Ng, Szu Hui, 2013. "A sequential approach for stochastic computer model calibration and prediction," Reliability Engineering and System Safety, Elsevier, vol. 111(C), pages 273-286.
    3. White, Staci A. & Herbei, Radu, 2015. "A Monte Carlo approach to quantifying model error in Bayesian parameter estimation," Computational Statistics & Data Analysis, Elsevier, vol. 83(C), pages 168-181.
    4. Wu, Xu & Kozlowski, Tomasz & Meidani, Hadi, 2018. "Kriging-based inverse uncertainty quantification of nuclear fuel performance code BISON fission gas release model using time series measurement data," Reliability Engineering and System Safety, Elsevier, vol. 169(C), pages 422-436.
    5. Paulo, Rui & García-Donato, Gonzalo & Palomo, Jesús, 2012. "Calibration of computer models with multivariate output," Computational Statistics & Data Analysis, Elsevier, vol. 56(12), pages 3959-3974.
    6. Matthew Plumlee, 2014. "Fast Prediction of Deterministic Functions Using Sparse Grid Experimental Designs," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(508), pages 1581-1591, December.
    7. Daniel W. Gladish & Daniel E. Pagendam & Luk J. M. Peeters & Petra M. Kuhnert & Jai Vaze, 2018. "Emulation Engines: Choice and Quantification of Uncertainty for Complex Hydrological Models," Journal of Agricultural, Biological and Environmental Statistics, Springer;The International Biometric Society;American Statistical Association, vol. 23(1), pages 39-62, March.
    8. Chen, Yewen & Chang, Xiaohui & Luo, Fangzhi & Huang, Hui, 2023. "Additive dynamic models for correcting numerical model outputs," Computational Statistics & Data Analysis, Elsevier, vol. 187(C).
    9. Bruno Sansó & Chris Forest, 2009. "Statistical calibration of climate system properties," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 58(4), pages 485-503, September.
    10. Wilkinson Richard David, 2013. "Approximate Bayesian computation (ABC) gives exact results under the assumption of model error," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 12(2), pages 129-141, May.
    11. Antony M. Overstall & David C. Woods, 2013. "A Strategy for Bayesian Inference for Computationally Expensive Models with Application to the Estimation of Stem Cell Properties," Biometrics, The International Biometric Society, vol. 69(2), pages 458-468, June.
    12. Won Chang & Murali Haran & Patrick Applegate & David Pollard, 2016. "Calibrating an Ice Sheet Model Using High-Dimensional Binary Spatial Data," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(513), pages 57-72, March.
    13. Mohammadi, Hossein & Challenor, Peter & Goodfellow, Marc, 2019. "Emulating dynamic non-linear simulators using Gaussian processes," Computational Statistics & Data Analysis, Elsevier, vol. 139(C), pages 178-196.
    14. Williams, Brian J. & Loeppky, Jason L. & Moore, Leslie M. & Macklem, Mason S., 2011. "Batch sequential design to achieve predictive maturity with calibrated computer models," Reliability Engineering and System Safety, Elsevier, vol. 96(9), pages 1208-1219.
    15. Luca Aiello & Matteo Fontana & Alessandra Guglielmi, 2023. "Bayesian functional emulation of CO2 emissions on future climate change scenarios," Environmetrics, John Wiley & Sons, Ltd., vol. 34(8), December.
    16. Jung, Yongsu & Lee, Ikjin, 2021. "Optimal design of experiments for optimization-based model calibration using Fisher information matrix," Reliability Engineering and System Safety, Elsevier, vol. 216(C).
    17. Kim, Wongon & Lee, Guesuk & Son, Hyejeong & Choi, Hyunhee & Youn, Byeng D., 2022. "Estimation of fatigue crack initiation and growth in engineering product development using a digital twin approach," Reliability Engineering and System Safety, Elsevier, vol. 226(C).
    18. Giri Gopalan & Christopher K. Wikle, 2022. "A Higher-Order Singular Value Decomposition Tensor Emulator for Spatiotemporal Simulators," Journal of Agricultural, Biological and Environmental Statistics, Springer;The International Biometric Society;American Statistical Association, vol. 27(1), pages 22-45, March.
    19. Maupin, Kathryn A. & Swiler, Laura P., 2020. "Model discrepancy calibration across experimental settings," Reliability Engineering and System Safety, Elsevier, vol. 200(C).
    20. Curtis B. Storlie & William A. Lane & Emily M. Ryan & James R. Gattiker & David M. Higdon, 2015. "Calibration of Computational Models With Categorical Parameters and Correlated Outputs via Bayesian Smoothing Spline ANOVA," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(509), pages 68-82, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:113:y:2017:i:c:p:346-362. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.