IDEAS home Printed from https://ideas.repec.org/p/msh/ebswps/2019-13.html
   My bibliography  Save this paper

Updating Variational Bayes: Fast Sequential Posterior Inference

Author

Listed:
  • Nathaniel Tomasetti
  • Catherine Forbes
  • Anastasios Panagiotelis

Abstract

Variational Bayesian (VB) methods usually produce posterior inference in a time frame considerably smaller than traditional Markov Chain Monte Carlo approaches. Although the VB posterior is an approximation, it has been shown to produce good parameter estimates and predicted values when a rich class of approximating distributions are considered. In this paper we propose Updating VB (UVB), a recursive algorithm used to update a sequence of VB posterior approximations in an online setting, with the computation of each posterior update requiring only the data observed since the previous update. An extension to the proposed algorithm, named UVB-IS, allows the user to trade accuracy for a substantial increase in computational speed through the use of importance sampling. The two methods and their properties are detailed in two separate simulation studies. Two empirical illustrations of the proposed UVB methods are provided, including one where a Dirichlet Process Mixture model is repeatedly updated in the context of predicting the future behaviour of vehicles on a stretch of the US Highway 101.

Suggested Citation

  • Nathaniel Tomasetti & Catherine Forbes & Anastasios Panagiotelis, 2019. "Updating Variational Bayes: Fast Sequential Posterior Inference," Monash Econometrics and Business Statistics Working Papers 13/19, Monash University, Department of Econometrics and Business Statistics.
    • Handle: RePEc:msh:ebswps:2019-13
    as

    Download full text from publisher

    File URL: https://www.monash.edu/business/ebs/research/publications/ebs/wp13-2019.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. David M. Blei & Alp Kucukelbir & Jon D. McAuliffe, 2017. "Variational Inference: A Review for Statisticians," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(518), pages 859-877, April.
    2. Bhattacharya, Arnab & Wilson, Simon P., 2018. "Sequential Bayesian inference for static parameters in dynamic state space models," Computational Statistics & Data Analysis, Elsevier, vol. 127(C), pages 187-203.
    3. Dißmann, J. & Brechmann, E.C. & Czado, C. & Kurowicka, D., 2013. "Selecting and estimating regular vine copulae and application to financial returns," Computational Statistics & Data Analysis, Elsevier, vol. 59(C), pages 52-69.
    4. Nicolas Chopin, 2002. "A sequential particle filter method for static models," Biometrika, Biometrika Trust, vol. 89(3), pages 539-552, August.
    5. W. R. Gilks & N. G. Best & K. K. C. Tan, 1995. "Adaptive Rejection Metropolis Sampling Within Gibbs Sampling," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 44(4), pages 455-472, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Loaiza-Maya, Rubén & Smith, Michael Stanley & Nott, David J. & Danaher, Peter J., 2022. "Fast and accurate variational inference for models with many latent variables," Journal of Econometrics, Elsevier, vol. 230(2), pages 339-362.
    2. Gunawan, David & Kohn, Robert & Nott, David, 2021. "Variational Bayes approximation of factor stochastic volatility models," International Journal of Forecasting, Elsevier, vol. 37(4), pages 1355-1375.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ho, Paul, 2023. "Global robust Bayesian analysis in large models," Journal of Econometrics, Elsevier, vol. 235(2), pages 608-642.
    2. Benedikt Schamberger & Lutz F. Gruber & Claudia Czado, 2017. "Bayesian Inference for Latent Factor Copulas and Application to Financial Risk Forecasting," Econometrics, MDPI, vol. 5(2), pages 1-23, May.
    3. Roger M. Cooke & Harry Joe & Bo Chang, 2020. "Vine copula regression for observational studies," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 104(2), pages 141-167, June.
    4. Cao, Juan & Zhang, Zhao & Tao, Fulu & Chen, Yi & Luo, Xiangzhong & Xie, Jun, 2023. "Forecasting global crop yields based on El Nino Southern Oscillation early signals," Agricultural Systems, Elsevier, vol. 205(C).
    5. England, Peter, 2002. "Addendum to "Analytic and bootstrap estimates of prediction errors in claims reserving"," Insurance: Mathematics and Economics, Elsevier, vol. 31(3), pages 461-466, December.
    6. Shen Liu & Hongyan Liu, 2021. "Tagging Items Automatically Based on Both Content Information and Browsing Behaviors," INFORMS Journal on Computing, INFORMS, vol. 33(3), pages 882-897, July.
    7. Mevin B. Hooten & Michael R. Schwob & Devin S. Johnson & Jacob S. Ivan, 2023. "Multistage hierarchical capture–recapture models," Environmetrics, John Wiley & Sons, Ltd., vol. 34(6), September.
    8. Wang, Mengjiao & Liu, Jianxu & Yang, Bing, 2024. "Does the strength of the US dollar affect the interdependence among currency exchange rates of RCEP and CPTPP countries?," Finance Research Letters, Elsevier, vol. 62(PA).
    9. Arnaud Dufays, 2016. "Evolutionary Sequential Monte Carlo Samplers for Change-Point Models," Econometrics, MDPI, vol. 4(1), pages 1-33, March.
    10. Jeong Eun Lee & Christian Robert, 2013. "Imortance Sampling Schemes for Evidence Approximation in Mixture Models," Working Papers 2013-42, Center for Research in Economics and Statistics.
    11. Loaiza-Maya, Rubén & Smith, Michael Stanley & Nott, David J. & Danaher, Peter J., 2022. "Fast and accurate variational inference for models with many latent variables," Journal of Econometrics, Elsevier, vol. 230(2), pages 339-362.
    12. Luo, Nanyu & Ji, Feng & Han, Yuting & He, Jinbo & Zhang, Xiaoya, 2024. "Fitting item response theory models using deep learning computational frameworks," OSF Preprints tjxab, Center for Open Science.
    13. James Martin & Ajay Jasra & Emma McCoy, 2013. "Inference for a class of partially observed point process models," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 65(3), pages 413-437, June.
    14. Zhi, Bangdong & Wang, Xiaojun & Xu, Fangming, 2022. "Managing inventory financing in a volatile market: A novel data-driven copula model," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 165(C).
    15. Xing Qin & Shuangge Ma & Mengyun Wu, 2023. "Two‐level Bayesian interaction analysis for survival data incorporating pathway information," Biometrics, The International Biometric Society, vol. 79(3), pages 1761-1774, September.
    16. Youngseon Lee & Seongil Jo & Jaeyong Lee, 2022. "A variational inference for the Lévy adaptive regression with multiple kernels," Computational Statistics, Springer, vol. 37(5), pages 2493-2515, November.
    17. Liu, Jie & Ye, Zifeng & Chen, Kun & Zhang, Panpan, 2024. "Variational Bayesian inference for bipartite mixed-membership stochastic block model with applications to collaborative filtering," Computational Statistics & Data Analysis, Elsevier, vol. 189(C).
    18. Gabriele Fiorentini & Enrique Sentana & Neil Shephard, 2004. "Likelihood-Based Estimation of Latent Generalized ARCH Structures," Econometrica, Econometric Society, vol. 72(5), pages 1481-1517, September.
    19. Gael M. Martin & David T. Frazier & Christian P. Robert, 2020. "Computing Bayes: Bayesian Computation from 1763 to the 21st Century," Monash Econometrics and Business Statistics Working Papers 14/20, Monash University, Department of Econometrics and Business Statistics.
    20. Djohan Bonnet & Tifenn Hirtzlin & Atreya Majumdar & Thomas Dalgaty & Eduardo Esmanhotto & Valentina Meli & Niccolo Castellani & Simon Martin & Jean-François Nodin & Guillaume Bourgeois & Jean-Michel P, 2023. "Bringing uncertainty quantification to the extreme-edge with memristor-based Bayesian neural networks," Nature Communications, Nature, vol. 14(1), pages 1-13, December.

    More about this item

    Keywords

    importance sampling; forecasting; clustering; Dirichlet process mixture; variational inference;
    All these keywords.

    JEL classification:

    • C11 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Bayesian Analysis: General
    • G18 - Financial Economics - - General Financial Markets - - - Government Policy and Regulation
    • G39 - Financial Economics - - Corporate Finance and Governance - - - Other

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:msh:ebswps:2019-13. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Professor Xibin Zhang (email available below). General contact details of provider: https://edirc.repec.org/data/dxmonau.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.