IDEAS home Printed from https://ideas.repec.org/a/taf/jnlasa/v110y2015i509p68-82.html
   My bibliography  Save this article

Calibration of Computational Models With Categorical Parameters and Correlated Outputs via Bayesian Smoothing Spline ANOVA

Author

Listed:
  • Curtis B. Storlie
  • William A. Lane
  • Emily M. Ryan
  • James R. Gattiker
  • David M. Higdon

Abstract

It has become commonplace to use complex computer models to predict outcomes in regions where data do not exist. Typically these models need to be calibrated and validated using some experimental data, which often consists of multiple correlated outcomes. In addition, some of the model parameters may be categorical in nature, such as a pointer variable to alternate models (or submodels) for some of the physics of the system. Here, we present a general approach for calibration in such situations where an emulator of the computationally demanding models and a discrepancy term from the model to reality are represented within a Bayesian smoothing spline (BSS) ANOVA framework. The BSS-ANOVA framework has several advantages over the traditional Gaussian process, including ease of handling categorical inputs and correlated outputs, and improved computational efficiency. Finally, this framework is then applied to the problem that motivated its design; a calibration of a computational fluid dynamics (CFD) model of a bubbling fluidized which is used as an absorber in a CO 2 capture system. Supplementary materials for this article are available online.

Suggested Citation

  • Curtis B. Storlie & William A. Lane & Emily M. Ryan & James R. Gattiker & David M. Higdon, 2015. "Calibration of Computational Models With Categorical Parameters and Correlated Outputs via Bayesian Smoothing Spline ANOVA," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(509), pages 68-82, March.
    • Handle: RePEc:taf:jnlasa:v:110:y:2015:i:509:p:68-82
    DOI: 10.1080/01621459.2014.979993
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1080/01621459.2014.979993
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1080/01621459.2014.979993?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Higdon, Dave & Gattiker, James & Williams, Brian & Rightley, Maria, 2008. "Computer Model Calibration Using High-Dimensional Output," Journal of the American Statistical Association, American Statistical Association, vol. 103, pages 570-583, June.
    2. Storlie, Curtis B. & Reich, Brian J. & Helton, Jon C. & Swiler, Laura P. & Sallaberry, Cedric J., 2013. "Analysis of computationally demanding models with continuous and categorical inputs," Reliability Engineering and System Safety, Elsevier, vol. 113(C), pages 30-41.
    3. Marc C. Kennedy & Anthony O'Hagan, 2001. "Bayesian calibration of computer models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 63(3), pages 425-464.
    4. Storlie, Curtis B. & Swiler, Laura P. & Helton, Jon C. & Sallaberry, Cedric J., 2009. "Implementation and evaluation of nonparametric regression procedures for sensitivity analysis of computationally demanding models," Reliability Engineering and System Safety, Elsevier, vol. 94(11), pages 1735-1763.
    5. Jeremy E. Oakley & Anthony O'Hagan, 2004. "Probabilistic sensitivity analysis of complex models: a Bayesian approach," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 66(3), pages 751-769, August.
    6. Helton, J.C. & Johnson, J.D. & Sallaberry, C.J. & Storlie, C.B., 2006. "Survey of sampling-based methods for uncertainty and sensitivity analysis," Reliability Engineering and System Safety, Elsevier, vol. 91(10), pages 1175-1209.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. K. Sham Bhat & David S. Mebane & Priyadarshi Mahapatra & Curtis B. Storlie, 2017. "Upscaling Uncertainty with Dynamic Discrepancy for a Multi-Scale Carbon Capture System," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(520), pages 1453-1467, October.
    2. Bledar A. Konomi & Georgios Karagiannis & Kevin Lai & Guang Lin, 2017. "Bayesian Treed Calibration: An Application to Carbon Capture With AX Sorbent," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(517), pages 37-53, January.
    3. Chao Wang & Zhijie Xu & Canhai Lai & Greg Whyatt & Peter Marcy & Xin Sun, 2017. "Hierarchical calibration and validation for modeling bench‐scale solvent‐based carbon capture. Part 1: Non‐reactive physical mass transfer across the wetted wall column," Greenhouse Gases: Science and Technology, Blackwell Publishing, vol. 7(4), pages 706-720, August.
    4. Raymond K. W. Wong & Curtis B. Storlie & Thomas C. M. Lee, 2017. "A frequentist approach to computer model calibration," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(2), pages 635-648, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. K. Sham Bhat & David S. Mebane & Priyadarshi Mahapatra & Curtis B. Storlie, 2017. "Upscaling Uncertainty with Dynamic Discrepancy for a Multi-Scale Carbon Capture System," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(520), pages 1453-1467, October.
    2. Daniel W. Gladish & Daniel E. Pagendam & Luk J. M. Peeters & Petra M. Kuhnert & Jai Vaze, 2018. "Emulation Engines: Choice and Quantification of Uncertainty for Complex Hydrological Models," Journal of Agricultural, Biological and Environmental Statistics, Springer;The International Biometric Society;American Statistical Association, vol. 23(1), pages 39-62, March.
    3. Storlie, Curtis B. & Reich, Brian J. & Helton, Jon C. & Swiler, Laura P. & Sallaberry, Cedric J., 2013. "Analysis of computationally demanding models with continuous and categorical inputs," Reliability Engineering and System Safety, Elsevier, vol. 113(C), pages 30-41.
    4. Raymond K. W. Wong & Curtis B. Storlie & Thomas C. M. Lee, 2017. "A frequentist approach to computer model calibration," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(2), pages 635-648, March.
    5. Wei, Pengfei & Lu, Zhenzhou & Song, Jingwen, 2015. "Variable importance analysis: A comprehensive review," Reliability Engineering and System Safety, Elsevier, vol. 142(C), pages 399-432.
    6. Borgonovo, Emanuele & Plischke, Elmar, 2016. "Sensitivity analysis: A review of recent advances," European Journal of Operational Research, Elsevier, vol. 248(3), pages 869-887.
    7. Daneshkhah, Alireza & Bedford, Tim, 2013. "Probabilistic sensitivity analysis of system availability using Gaussian processes," Reliability Engineering and System Safety, Elsevier, vol. 112(C), pages 82-93.
    8. Blatman, Géraud & Sudret, Bruno, 2010. "Efficient computation of global sensitivity indices using sparse polynomial chaos expansions," Reliability Engineering and System Safety, Elsevier, vol. 95(11), pages 1216-1229.
    9. Daniel W. Gladish & Ross Darnell & Peter J. Thorburn & Bhakti Haldankar, 2019. "Emulated Multivariate Global Sensitivity Analysis for Complex Computer Models Applied to Agricultural Simulators," Journal of Agricultural, Biological and Environmental Statistics, Springer;The International Biometric Society;American Statistical Association, vol. 24(1), pages 130-153, March.
    10. Ioannis Andrianakis & Ian R Vernon & Nicky McCreesh & Trevelyan J McKinley & Jeremy E Oakley & Rebecca N Nsubuga & Michael Goldstein & Richard G White, 2015. "Bayesian History Matching of Complex Infectious Disease Models Using Emulation: A Tutorial and a Case Study on HIV in Uganda," PLOS Computational Biology, Public Library of Science, vol. 11(1), pages 1-18, January.
    11. Emanuele Borgonovo & Gordon B. Hazen & Elmar Plischke, 2016. "A Common Rationale for Global Sensitivity Measures and Their Estimation," Risk Analysis, John Wiley & Sons, vol. 36(10), pages 1871-1895, October.
    12. Tian, Wei, 2013. "A review of sensitivity analysis methods in building energy analysis," Renewable and Sustainable Energy Reviews, Elsevier, vol. 20(C), pages 411-419.
    13. Di Maio, Francesco & Bandini, Alessandro & Zio, Enrico & Alberola, Sofia Carlos & Sanchez-Saez, Francisco & Martorell, Sebastián, 2016. "Bootstrapped-ensemble-based Sensitivity Analysis of a trace thermal-hydraulic model based on a limited number of PWR large break loca simulations," Reliability Engineering and System Safety, Elsevier, vol. 153(C), pages 122-134.
    14. Manfren, Massimiliano & Aste, Niccolò & Moshksar, Reza, 2013. "Calibration and uncertainty analysis for computer models – A meta-model based approach for integrated building energy simulation," Applied Energy, Elsevier, vol. 103(C), pages 627-641.
    15. Nagel, Joseph B. & Rieckermann, Jörg & Sudret, Bruno, 2020. "Principal component analysis and sparse polynomial chaos expansions for global sensitivity analysis and model calibration: Application to urban drainage simulation," Reliability Engineering and System Safety, Elsevier, vol. 195(C).
    16. Zhai, Qingqing & Yang, Jun & Xie, Min & Zhao, Yu, 2014. "Generalized moment-independent importance measures based on Minkowski distance," European Journal of Operational Research, Elsevier, vol. 239(2), pages 449-455.
    17. Francesco Di Maio & Nicola Pedroni & Barnabás Tóth & Luciano Burgazzi & Enrico Zio, 2021. "Reliability Assessment of Passive Safety Systems for Nuclear Energy Applications: State-of-the-Art and Open Issues," Energies, MDPI, vol. 14(15), pages 1-17, August.
    18. Allaire, Douglas L. & Willcox, Karen E., 2012. "A variance-based sensitivity index function for factor prioritization," Reliability Engineering and System Safety, Elsevier, vol. 107(C), pages 107-114.
    19. Storlie, Curtis B. & Swiler, Laura P. & Helton, Jon C. & Sallaberry, Cedric J., 2009. "Implementation and evaluation of nonparametric regression procedures for sensitivity analysis of computationally demanding models," Reliability Engineering and System Safety, Elsevier, vol. 94(11), pages 1735-1763.
    20. Hansen, Clifford W. & Helton, Jon C. & Sallaberry, Cédric J., 2012. "Use of replicated Latin hypercube sampling to estimate sampling variance in uncertainty and sensitivity analysis results for the geologic disposal of radioactive waste," Reliability Engineering and System Safety, Elsevier, vol. 107(C), pages 139-148.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:taf:jnlasa:v:110:y:2015:i:509:p:68-82. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Longhurst (email available below). General contact details of provider: http://www.tandfonline.com/UASA20 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.