IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v71y2014icp491-505.html
   My bibliography  Save this article

Mixtures of experts for understanding model discrepancy in dynamic computer models

Author

Listed:
  • Nott, David J.
  • Marshall, Lucy
  • Fielding, Mark
  • Liong, Shie-Yui

Abstract

There are many areas of science and engineering where research and decision making are performed using computer models. These computer models are usually deterministic and may take minutes, hours or days to produce an output for a single value of the model inputs. Fitting mixtures of experts of computer models where the expert components use different values of the computer model parameters is considered. The efficient calibration of such models using emulators, which are fast statistical surrogates for the computer model, is discussed. It is argued that mixtures of experts are often insightful for describing model discrepancy and ways in which the computer model can be improved. This is not a strength of standard approaches to the statistical analysis of computer models where a certain “best input” assumption is usually made and model discrepancy is often described through a stationary Gaussian process prior on the discrepancy function. Application of the framework is presented for a dynamic hydrological rainfall–runoff model in which the mixture approach is helpful for highlighting model deficiencies.

Suggested Citation

  • Nott, David J. & Marshall, Lucy & Fielding, Mark & Liong, Shie-Yui, 2014. "Mixtures of experts for understanding model discrepancy in dynamic computer models," Computational Statistics & Data Analysis, Elsevier, vol. 71(C), pages 491-505.
    • Handle: RePEc:eee:csdana:v:71:y:2014:i:c:p:491-505
    DOI: 10.1016/j.csda.2013.04.020
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167947313001631
    Download Restriction: Full text for ScienceDirect subscribers only.
    File URL: https://libkey.io/10.1016/j.csda.2013.04.020?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. S. Conti & J. P. Gosling & J. E. Oakley & A. O'Hagan, 2009. "Gaussian process emulation of dynamic computer codes," Biometrika, Biometrika Trust, vol. 96(3), pages 663-676.
    2. Reichert, P. & White, G. & Bayarri, M.J. & Pitman, E.B., 2011. "Mechanism-based emulation of dynamic simulation models: Concept and application in hydrology," Computational Statistics & Data Analysis, Elsevier, vol. 55(4), pages 1638-1655, April.
    3. J. Durbin, 2002. "A simple and efficient simulation smoother for state space time series analysis," Biometrika, Biometrika Trust, vol. 89(3), pages 603-616, August.
    4. Alexandre X. Carvalho & Martin A. Tanner, 2006. "Modeling nonlinearities with mixtures-of-experts of time series models," International Journal of Mathematics and Mathematical Sciences, Hindawi, vol. 2006, pages 1-22, August.
    5. Goldstein, Michael & Rougier, Jonathan, 2006. "Bayes Linear Calibrated Prediction for Complex Systems," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 1132-1143, September.
    6. Higdon, Dave & Gattiker, James & Williams, Brian & Rightley, Maria, 2008. "Computer Model Calibration Using High-Dimensional Output," Journal of the American Statistical Association, American Statistical Association, vol. 103, pages 570-583, June.
    7. Marc C. Kennedy & Anthony O'Hagan, 2001. "Bayesian calibration of computer models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 63(3), pages 425-464.
    8. Craig P. S & Goldstein M. & Rougier J. C & Seheult A. H, 2001. "Bayesian Forecasting for Complex Systems Using Computer Simulators," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 717-729, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mohammadi, Hossein & Challenor, Peter & Goodfellow, Marc, 2019. "Emulating dynamic non-linear simulators using Gaussian processes," Computational Statistics & Data Analysis, Elsevier, vol. 139(C), pages 178-196.
    2. Jackson Samuel E. & Vernon Ian & Liu Junli & Lindsey Keith, 2020. "Understanding hormonal crosstalk in Arabidopsis root development via emulation and history matching," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 19(2), pages 1-33, April.
    3. Paulo, Rui & García-Donato, Gonzalo & Palomo, Jesús, 2012. "Calibration of computer models with multivariate output," Computational Statistics & Data Analysis, Elsevier, vol. 56(12), pages 3959-3974.
    4. Daniel W. Gladish & Daniel E. Pagendam & Luk J. M. Peeters & Petra M. Kuhnert & Jai Vaze, 2018. "Emulation Engines: Choice and Quantification of Uncertainty for Complex Hydrological Models," Journal of Agricultural, Biological and Environmental Statistics, Springer;The International Biometric Society;American Statistical Association, vol. 23(1), pages 39-62, March.
    5. Ioannis Andrianakis & Ian R Vernon & Nicky McCreesh & Trevelyan J McKinley & Jeremy E Oakley & Rebecca N Nsubuga & Michael Goldstein & Richard G White, 2015. "Bayesian History Matching of Complex Infectious Disease Models Using Emulation: A Tutorial and a Case Study on HIV in Uganda," PLOS Computational Biology, Public Library of Science, vol. 11(1), pages 1-18, January.
    6. Reichert, P. & White, G. & Bayarri, M.J. & Pitman, E.B., 2011. "Mechanism-based emulation of dynamic simulation models: Concept and application in hydrology," Computational Statistics & Data Analysis, Elsevier, vol. 55(4), pages 1638-1655, April.
    7. Xiaoyu Xiong & Benjamin D. Youngman & Theodoros Economou, 2021. "Data fusion with Gaussian processes for estimation of environmental hazard events," Environmetrics, John Wiley & Sons, Ltd., vol. 32(3), May.
    8. Drignei, Dorin, 2011. "A general statistical model for computer experiments with time series output," Reliability Engineering and System Safety, Elsevier, vol. 96(4), pages 460-467.
    9. Campbell, Katherine, 2006. "Statistical calibration of computer simulations," Reliability Engineering and System Safety, Elsevier, vol. 91(10), pages 1358-1363.
    10. White, Staci A. & Herbei, Radu, 2015. "A Monte Carlo approach to quantifying model error in Bayesian parameter estimation," Computational Statistics & Data Analysis, Elsevier, vol. 83(C), pages 168-181.
    11. Wu, Xu & Kozlowski, Tomasz & Meidani, Hadi, 2018. "Kriging-based inverse uncertainty quantification of nuclear fuel performance code BISON fission gas release model using time series measurement data," Reliability Engineering and System Safety, Elsevier, vol. 169(C), pages 422-436.
    12. Matthew Plumlee, 2014. "Fast Prediction of Deterministic Functions Using Sparse Grid Experimental Designs," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(508), pages 1581-1591, December.
    13. Chen, Yewen & Chang, Xiaohui & Luo, Fangzhi & Huang, Hui, 2023. "Additive dynamic models for correcting numerical model outputs," Computational Statistics & Data Analysis, Elsevier, vol. 187(C).
    14. Bruno Sansó & Chris Forest, 2009. "Statistical calibration of climate system properties," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 58(4), pages 485-503, September.
    15. Wilkinson Richard David, 2013. "Approximate Bayesian computation (ABC) gives exact results under the assumption of model error," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 12(2), pages 129-141, May.
    16. Antony M. Overstall & David C. Woods, 2013. "A Strategy for Bayesian Inference for Computationally Expensive Models with Application to the Estimation of Stem Cell Properties," Biometrics, The International Biometric Society, vol. 69(2), pages 458-468, June.
    17. Won Chang & Murali Haran & Patrick Applegate & David Pollard, 2016. "Calibrating an Ice Sheet Model Using High-Dimensional Binary Spatial Data," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(513), pages 57-72, March.
    18. Williams, Brian J. & Loeppky, Jason L. & Moore, Leslie M. & Macklem, Mason S., 2011. "Batch sequential design to achieve predictive maturity with calibrated computer models," Reliability Engineering and System Safety, Elsevier, vol. 96(9), pages 1208-1219.
    19. Luca Aiello & Matteo Fontana & Alessandra Guglielmi, 2023. "Bayesian functional emulation of CO2 emissions on future climate change scenarios," Environmetrics, John Wiley & Sons, Ltd., vol. 34(8), December.
    20. Jung, Yongsu & Lee, Ikjin, 2021. "Optimal design of experiments for optimization-based model calibration using Fisher information matrix," Reliability Engineering and System Safety, Elsevier, vol. 216(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:71:y:2014:i:c:p:491-505. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.