IDEAS home Printed from https://ideas.repec.org/a/bla/jorssb/v84y2022i4p1198-1228.html
   My bibliography  Save this article

Fast increased fidelity samplers for approximate Bayesian Gaussian process regression

Author

Listed:
  • Kelly R. Moran
  • Matthew W. Wheeler

Abstract

Gaussian processes (GPs) are common components in Bayesian non‐parametric models having a rich methodological literature and strong theoretical grounding. The use of exact GPs in Bayesian models is limited to problems containing several thousand observations due to their prohibitive computational demands. We develop a posterior sampling algorithm using H‐matrix approximations that scales at O(nlog2n). We show that this approximation's Kullback–Leibler divergence to the true posterior can be made arbitrarily small. Although multidimensional GPs could be used with our algorithm, d‐dimensional surfaces are modelled as tensor products of univariate GPs to minimize the cost of matrix construction and maximize computational efficiency. We illustrate the performance of this fast increased fidelity approximate GP, FIFA‐GP, using both simulated and non‐synthetic data sets.

Suggested Citation

  • Kelly R. Moran & Matthew W. Wheeler, 2022. "Fast increased fidelity samplers for approximate Bayesian Gaussian process regression," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 84(4), pages 1198-1228, September.
    • Handle: RePEc:bla:jorssb:v:84:y:2022:i:4:p:1198-1228
    DOI: 10.1111/rssb.12494
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/rssb.12494
    Download Restriction: no
    File URL: https://libkey.io/10.1111/rssb.12494?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Carlos M. Carvalho & Nicholas G. Polson & James G. Scott, 2010. "The horseshoe estimator for sparse signals," Biometrika, Biometrika Trust, vol. 97(2), pages 465-480.
    2. Abhirup Datta & Sudipto Banerjee & Andrew O. Finley & Alan E. Gelfand, 2016. "Hierarchical Nearest-Neighbor Gaussian Process Models for Large Geostatistical Datasets," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(514), pages 800-812, April.
    3. Sudipto Banerjee & Alan E. Gelfand & Andrew O. Finley & Huiyan Sang, 2008. "Gaussian predictive process models for large spatial data sets," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 70(4), pages 825-848, September.
    4. Litvinenko, Alexander & Sun, Ying & Genton, Marc G. & Keyes, David E., 2019. "Likelihood approximation with hierarchical matrices for large spatial datasets," Computational Statistics & Data Analysis, Elsevier, vol. 137(C), pages 115-132.
    5. Antonio Canale & David B. Dunson, 2013. "Nonparametric Bayes modelling of count processes," Biometrika, Biometrika Trust, vol. 100(4), pages 801-816.
    6. Gramacy, Robert B & Lee, Herbert K. H, 2008. "Bayesian Treed Gaussian Process Models With an Application to Computer Modeling," Journal of the American Statistical Association, American Statistical Association, vol. 103(483), pages 1119-1130.
    7. Huiyan Sang & Jianhua Z. Huang, 2012. "A full scale approximation of covariance functions for large spatial data sets," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 74(1), pages 111-132, January.
    8. Anjishnu Banerjee & David B. Dunson & Surya T. Tokdar, 2013. "Efficient Gaussian process regression for large datasets," Biometrika, Biometrika Trust, vol. 100(1), pages 75-89.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Huang Huang & Sameh Abdulah & Ying Sun & Hatem Ltaief & David E. Keyes & Marc G. Genton, 2021. "Competition on Spatial Statistics for Large Datasets," Journal of Agricultural, Biological and Environmental Statistics, Springer;The International Biometric Society;American Statistical Association, vol. 26(4), pages 580-595, December.
    2. Isabelle Grenier & Bruno Sansó & Jessica L. Matthews, 2024. "Multivariate nearest‐neighbors Gaussian processes with random covariance matrices," Environmetrics, John Wiley & Sons, Ltd., vol. 35(3), May.
    3. Marchetti, Yuliya & Nguyen, Hai & Braverman, Amy & Cressie, Noel, 2018. "Spatial data compression via adaptive dispersion clustering," Computational Statistics & Data Analysis, Elsevier, vol. 117(C), pages 138-153.
    4. Matthew J. Heaton & Abhirup Datta & Andrew O. Finley & Reinhard Furrer & Joseph Guinness & Rajarshi Guhaniyogi & Florian Gerber & Robert B. Gramacy & Dorit Hammerling & Matthias Katzfuss & Finn Lindgr, 2019. "A Case Study Competition Among Methods for Analyzing Large Spatial Data," Journal of Agricultural, Biological and Environmental Statistics, Springer;The International Biometric Society;American Statistical Association, vol. 24(3), pages 398-425, September.
    5. Si Cheng & Bledar A. Konomi & Georgios Karagiannis & Emily L. Kang, 2024. "Recursive nearest neighbor co‐kriging models for big multi‐fidelity spatial data sets," Environmetrics, John Wiley & Sons, Ltd., vol. 35(4), June.
    6. Jialuo Liu & Tingjin Chu & Jun Zhu & Haonan Wang, 2022. "Large spatial data modeling and analysis: A Krylov subspace approach," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 49(3), pages 1115-1143, September.
    7. Jingjing Yang & Dennis D. Cox & Jong Soo Lee & Peng Ren & Taeryon Choi, 2017. "Efficient Bayesian hierarchical functional data analysis with basis function approximations using Gaussian–Wishart processes," Biometrics, The International Biometric Society, vol. 73(4), pages 1082-1091, December.
    8. Cole, D. Austin & Gramacy, Robert B. & Ludkovski, Mike, 2022. "Large-scale local surrogate modeling of stochastic simulation experiments," Computational Statistics & Data Analysis, Elsevier, vol. 174(C).
    9. Bledar A. Konomi & Emily L. Kang & Ayat Almomani & Jonathan Hobbs, 2023. "Bayesian Latent Variable Co-kriging Model in Remote Sensing for Quality Flagged Observations," Journal of Agricultural, Biological and Environmental Statistics, Springer;The International Biometric Society;American Statistical Association, vol. 28(3), pages 423-441, September.
    10. Paciorek, Christopher J. & Lipshitz, Benjamin & Zhuo, Wei & Prabhat, . & Kaufman, Cari G. G. & Thomas, Rollin C., 2015. "Parallelizing Gaussian Process Calculations in R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 63(i10).
    11. Litvinenko, Alexander & Sun, Ying & Genton, Marc G. & Keyes, David E., 2019. "Likelihood approximation with hierarchical matrices for large spatial datasets," Computational Statistics & Data Analysis, Elsevier, vol. 137(C), pages 115-132.
    12. Shinichiro Shirota & Andrew O. Finley & Bruce D. Cook & Sudipto Banerjee, 2023. "Conjugate sparse plus low rank models for efficient Bayesian interpolation of large spatial data," Environmetrics, John Wiley & Sons, Ltd., vol. 34(1), February.
    13. Peter A. Gao & Hannah M. Director & Cecilia M. Bitz & Adrian E. Raftery, 2022. "Probabilistic Forecasts of Arctic Sea Ice Thickness," Journal of Agricultural, Biological and Environmental Statistics, Springer;The International Biometric Society;American Statistical Association, vol. 27(2), pages 280-302, June.
    14. Waley W. J. Liang & Herbert K. H. Lee, 2019. "Bayesian nonstationary Gaussian process models via treed process convolutions," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 13(3), pages 797-818, September.
    15. Matthias Katzfuss & Joseph Guinness & Wenlong Gong & Daniel Zilber, 2020. "Vecchia Approximations of Gaussian-Process Predictions," Journal of Agricultural, Biological and Environmental Statistics, Springer;The International Biometric Society;American Statistical Association, vol. 25(3), pages 383-414, September.
    16. Matthew W. Wheeler, 2019. "Bayesian additive adaptive basis tensor product models for modeling high dimensional surfaces: an application to high‐throughput toxicity testing," Biometrics, The International Biometric Society, vol. 75(1), pages 193-201, March.
    17. Andrew Hoegh & Marco A. R. Ferreira & Scotland Leman, 2016. "Spatiotemporal model fusion: multiscale modelling of civil unrest," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 65(4), pages 529-545, August.
    18. Philip A. White & Alan E. Gelfand, 2021. "Multivariate functional data modeling with time-varying clustering," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 30(3), pages 586-602, September.
    19. Marco H. Benedetti & Veronica J. Berrocal & Naveen N. Narisetty, 2022. "Identifying regions of inhomogeneities in spatial processes via an M‐RA and mixture priors," Biometrics, The International Biometric Society, vol. 78(2), pages 798-811, June.
    20. Jaewoo Park & Sangwan Lee, 2022. "A projection‐based Laplace approximation for spatial latent variable models," Environmetrics, John Wiley & Sons, Ltd., vol. 33(1), February.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssb:v:84:y:2022:i:4:p:1198-1228. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/rssssea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.