IDEAS home Printed from https://ideas.repec.org/a/bla/jorssb/v82y2020i4p1087-1114.html
   My bibliography  Save this article

Quasi‐Bayes properties of a procedure for sequential learning in mixture models

Author

Listed:
  • Sandra Fortini
  • Sonia Petrone

Abstract

Bayesian methods are often optimal, yet increasing pressure for fast computations, especially with streaming data, brings renewed interest in faster, possibly suboptimal, solutions. The extent to which these algorithms approximate Bayesian solutions is a question of interest, but often unanswered. We propose a methodology to address this question in predictive settings, when the algorithm can be reinterpreted as a probabilistic predictive rule. We specifically develop the proposed methodology for a recursive procedure for on‐line learning in non‐parametric mixture models, which is often referred to as Newton's algorithm. This algorithm is simple and fast; however, its approximation properties are unclear. By reinterpreting it as a predictive rule, we can show that it underlies a statistical model which is, asymptotically, a Bayesian, exchangeable mixture model. In this sense, the recursive rule provides a quasi‐Bayes solution. Although the algorithm offers only a point estimate, our clean statistical formulation enables us to provide the asymptotic posterior distribution and asymptotic credible intervals for the mixing distribution. Moreover, it gives insights for tuning the parameters, as we illustrate in simulation studies, and paves the way to extensions in various directions. Beyond mixture models, our approach can be applied to other predictive algorithms.

Suggested Citation

  • Sandra Fortini & Sonia Petrone, 2020. "Quasi‐Bayes properties of a procedure for sequential learning in mixture models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 82(4), pages 1087-1114, September.
  • Handle: RePEc:bla:jorssb:v:82:y:2020:i:4:p:1087-1114
    DOI: 10.1111/rssb.12385
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/rssb.12385
    Download Restriction: no

    File URL: https://libkey.io/10.1111/rssb.12385?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. David M. Blei & Alp Kucukelbir & Jon D. McAuliffe, 2017. "Variational Inference: A Review for Statisticians," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(518), pages 859-877, April.
    2. Lorenzo Cappello & Stephen G. Walker, 2018. "A Bayesian Motivated Laplace Inversion for Multivariate Probability Distributions," Methodology and Computing in Applied Probability, Springer, vol. 20(2), pages 777-797, June.
    3. P. Richard Hahn & Ryan Martin & Stephen G. Walker, 2018. "On Recursive Bayesian Predictive Distributions," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(523), pages 1085-1093, July.
    4. Edoardo M. Airoldi & Thiago Costa & Federico Bassetti & Fabrizio Leisen & Michele Guindani, 2014. "Generalized Species Sampling Priors With Latent Beta Reinforcements," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(508), pages 1466-1480, December.
    5. Ryan Martin & Surya T. Tokdar, 2011. "Semiparametric inference in mixture models with predictive recursion marginal likelihood," Biometrika, Biometrika Trust, vol. 98(3), pages 567-582.
    6. Sonia Petrone & Piero Veronese, 2002. "Non parametric mixture priors based on an exponential random scheme," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 11(1), pages 1-20, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Sandra Fortini & Sonia Petrone & Hristo Sariev, 2021. "Predictive Constructions Based on Measure-Valued Pólya Urn Processes," Mathematics, MDPI, vol. 9(22), pages 1-19, November.
    2. Patrizia Berti & Luca Pratelli & Pietro Rigo, 2021. "A Central Limit Theorem for Predictive Distributions," Mathematics, MDPI, vol. 9(24), pages 1-11, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Patrizia Berti & Luca Pratelli & Pietro Rigo, 2021. "A Central Limit Theorem for Predictive Distributions," Mathematics, MDPI, vol. 9(24), pages 1-11, December.
    2. Jeong, Kuhwan & Chae, Minwoo & Kim, Yongdai, 2023. "Online learning for the Dirichlet process mixture model via weakly conjugate approximation," Computational Statistics & Data Analysis, Elsevier, vol. 179(C).
    3. Berti, Patrizia & Dreassi, Emanuela & Pratelli, Luca & Rigo, Pietro, 2021. "Asymptotics of certain conditionally identically distributed sequences," Statistics & Probability Letters, Elsevier, vol. 168(C).
    4. Ryan Martin, 2021. "A Survey of Nonparametric Mixing Density Estimation via the Predictive Recursion Algorithm," Sankhya B: The Indian Journal of Statistics, Springer;Indian Statistical Institute, vol. 83(1), pages 97-121, May.
    5. Shen Liu & Hongyan Liu, 2021. "Tagging Items Automatically Based on Both Content Information and Browsing Behaviors," INFORMS Journal on Computing, INFORMS, vol. 33(3), pages 882-897, July.
    6. Loaiza-Maya, Rubén & Smith, Michael Stanley & Nott, David J. & Danaher, Peter J., 2022. "Fast and accurate variational inference for models with many latent variables," Journal of Econometrics, Elsevier, vol. 230(2), pages 339-362.
    7. Lin, Edward M.H. & Sun, Edward W. & Yu, Min-Teh, 2020. "Behavioral data-driven analysis with Bayesian method for risk management of financial services," International Journal of Production Economics, Elsevier, vol. 228(C).
    8. Luo, Nanyu & Ji, Feng & Han, Yuting & He, Jinbo & Zhang, Xiaoya, 2024. "Fitting item response theory models using deep learning computational frameworks," OSF Preprints tjxab, Center for Open Science.
    9. Xing Qin & Shuangge Ma & Mengyun Wu, 2023. "Two‐level Bayesian interaction analysis for survival data incorporating pathway information," Biometrics, The International Biometric Society, vol. 79(3), pages 1761-1774, September.
    10. Youngseon Lee & Seongil Jo & Jaeyong Lee, 2022. "A variational inference for the Lévy adaptive regression with multiple kernels," Computational Statistics, Springer, vol. 37(5), pages 2493-2515, November.
    11. Liu, Jie & Ye, Zifeng & Chen, Kun & Zhang, Panpan, 2024. "Variational Bayesian inference for bipartite mixed-membership stochastic block model with applications to collaborative filtering," Computational Statistics & Data Analysis, Elsevier, vol. 189(C).
    12. Nathaniel Tomasetti & Catherine Forbes & Anastasios Panagiotelis, 2019. "Updating Variational Bayes: Fast Sequential Posterior Inference," Monash Econometrics and Business Statistics Working Papers 13/19, Monash University, Department of Econometrics and Business Statistics.
    13. Gael M. Martin & David T. Frazier & Christian P. Robert, 2020. "Computing Bayes: Bayesian Computation from 1763 to the 21st Century," Monash Econometrics and Business Statistics Working Papers 14/20, Monash University, Department of Econometrics and Business Statistics.
    14. Djohan Bonnet & Tifenn Hirtzlin & Atreya Majumdar & Thomas Dalgaty & Eduardo Esmanhotto & Valentina Meli & Niccolo Castellani & Simon Martin & Jean-François Nodin & Guillaume Bourgeois & Jean-Michel P, 2023. "Bringing uncertainty quantification to the extreme-edge with memristor-based Bayesian neural networks," Nature Communications, Nature, vol. 14(1), pages 1-13, December.
    15. Ho, Paul, 2023. "Global robust Bayesian analysis in large models," Journal of Econometrics, Elsevier, vol. 235(2), pages 608-642.
    16. Liang, Xinbin & Liu, Zhuoxuan & Wang, Jie & Jin, Xinqiao & Du, Zhimin, 2023. "Uncertainty quantification-based robust deep learning for building energy systems considering distribution shift problem," Applied Energy, Elsevier, vol. 337(C).
    17. Gael M. Martin & David T. Frazier & Ruben Loaiza-Maya & Florian Huber & Gary Koop & John Maheu & Didier Nibbering & Anastasios Panagiotelis, 2023. "Bayesian Forecasting in the 21st Century: A Modern Review," Monash Econometrics and Business Statistics Working Papers 1/23, Monash University, Department of Econometrics and Business Statistics.
    18. Seokhyun Chung & Raed Al Kontar & Zhenke Wu, 2022. "Weakly Supervised Multi-output Regression via Correlated Gaussian Processes," INFORMS Joural on Data Science, INFORMS, vol. 1(2), pages 115-137, October.
    19. Gary Koop & Dimitris Korobilis, 2023. "Bayesian Dynamic Variable Selection In High Dimensions," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 64(3), pages 1047-1074, August.
    20. Ziqi Zhang & Xinye Zhao & Mehak Bindra & Peng Qiu & Xiuwei Zhang, 2024. "scDisInFact: disentangled learning for integration and prediction of multi-batch multi-condition single-cell RNA-sequencing data," Nature Communications, Nature, vol. 15(1), pages 1-16, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssb:v:82:y:2020:i:4:p:1087-1114. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/rssssea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.