IDEAS home Printed from https://ideas.repec.org/p/zbw/caseps/200424.html
   My bibliography  Save this paper

The EM Algorithm

Author

Listed:
  • McLachlan, Geoffrey J.
  • Krishnan, Thriyambakam
  • Ng, See Ket

Abstract

The Expectation-Maximization (EM) algorithm is a broadly applicable approach to the iterative computation of maximum likelihood (ML) estimates, useful in a variety of incomplete-data problems. Maximum likelihood estimation and likelihood-based inference are of central importance in statistical theory and data analysis. Maximum likelihood estimation is a general-purpose method with attractive properties. It is the most-often used estimation technique in the frequentist framework; it is also relevant in the Bayesian framework (Chapter III.11). Often Bayesian solutions are justified with the help of likelihoods and maximum likelihood estimates (MLE), and Bayesian solutions are similar to penalized likelihood estimates. Maximum likelihood estimation is an ubiquitous technique and is used extensively in every area where statistical techniques are used.

Suggested Citation

  • McLachlan, Geoffrey J. & Krishnan, Thriyambakam & Ng, See Ket, 2004. "The EM Algorithm," Papers 2004,24, Humboldt University of Berlin, Center for Applied Statistics and Economics (CASE).
    • Handle: RePEc:zbw:caseps:200424
    as

    Download full text from publisher

    File URL: https://www.econstor.eu/bitstream/10419/22198/1/24_tk_gm_skn.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. J. G. Booth & J. P. Hobert, 1999. "Maximizing generalized linear mixed model likelihoods with an automated Monte Carlo EM algorithm," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 61(1), pages 265-285.
    2. McLachlan, G. J. & Peel, D. & Bean, R. W., 2003. "Modelling high-dimensional data by mixtures of factor analyzers," Computational Statistics & Data Analysis, Elsevier, vol. 41(3-4), pages 379-388, January.
    3. Robert, Christian P. & Celeux, Gilles & Diebolt, Jean, 1993. "Bayesian estimation of hidden Markov chains: a stochastic implementation," Statistics & Probability Letters, Elsevier, vol. 16(1), pages 77-83, January.
    4. M. Jamshidian & R. I. Jennrich, 2000. "Standard errors for EM estimation," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 62(2), pages 257-270.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. James C. Spall, 2012. "Cyclic Seesaw Process for Optimization and Identification," Journal of Optimization Theory and Applications, Springer, vol. 154(1), pages 187-208, July.
    2. Guillaume Horny, 2009. "Inference in mixed proportional hazard models with K random effects," Statistical Papers, Springer, vol. 50(3), pages 481-499, June.
    3. Jakubowski, Wojciech & Szulczewski, Wiesław & Żyromski, Andrzej & Biniak-Pieróg, Małgorzata, 2016. "The estimation of basket willow (Salix viminalis) yield – New approach, Part II: Theoretical model and its practical application," Renewable and Sustainable Energy Reviews, Elsevier, vol. 66(C), pages 843-851.
    4. Qunqiang Feng & Hosam Mahmoud & Alois Panholzer, 2008. "Limit laws for the Randić index of random binary tree models," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 60(2), pages 319-343, June.
    5. Ke-Hai Yuan & Kentaro Hayashi, 2005. "On muthén’s maximum likelihood for two-level covariance structure models," Psychometrika, Springer;The Psychometric Society, vol. 70(1), pages 147-167, March.
    6. Ringle, Christian M., 2006. "Segmentation for path models and unobserved heterogeneity: The finite mixture partial least squares approach," MPRA Paper 10734, University Library of Munich, Germany.
    7. Orozco-Garcia, Carolina & Schmeiser, Hato, 2015. "How sensitive is the pricing of lookback and interest rate guarantees when changing the modelling assumptions?," Insurance: Mathematics and Economics, Elsevier, vol. 65(C), pages 77-93.
    8. Saeedeh Eskandari & Mahdis Amiri & Nitheshnirmal Sãdhasivam & Hamid Reza Pourghasemi, 2020. "Comparison of new individual and hybrid machine learning algorithms for modeling and mapping fire hazard: a supplementary analysis of fire hazard in different counties of Golestan Province in Iran," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 104(1), pages 305-327, October.
    9. Ke-Hai Yuan & Peter Bentler, 2004. "On the asymptotic distributions of two statistics for two-level covariance structure models within the class of elliptical distributions," Psychometrika, Springer;The Psychometric Society, vol. 69(3), pages 437-457, September.
    10. Branislav Panić & Jernej Klemenc & Marko Nagode, 2020. "Improved Initialization of the EM Algorithm for Mixture Model Parameter Estimation," Mathematics, MDPI, vol. 8(3), pages 1-29, March.
    11. Christophe Genolini & Bruno Falissard, 2010. "KmL: k-means for longitudinal data," Computational Statistics, Springer, vol. 25(2), pages 317-328, June.
    12. Fordellone, Mario & Vichi, Maurizio, 2020. "Finding groups in structural equation modeling through the partial least squares algorithm," Computational Statistics & Data Analysis, Elsevier, vol. 147(C).
    13. Żyromski, Andrzej & Szulczewski, Wiesław & Biniak-Pieróg, Małgorzata & Jakubowski, Wojciech, 2016. "The estimation of basket willow (Salix viminalis) yield – New approach. Part I: Background and statistical description," Renewable and Sustainable Energy Reviews, Elsevier, vol. 65(C), pages 1118-1126.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hemant Kulkarni & Jayabrata Biswas & Kiranmoy Das, 2019. "A joint quantile regression model for multiple longitudinal outcomes," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 103(4), pages 453-473, December.
    2. Brent A. Coull & Alan Agresti, 2000. "Random Effects Modeling of Multiple Binomial Responses Using the Multivariate Binomial Logit-Normal Distribution," Biometrics, The International Biometric Society, vol. 56(1), pages 73-80, March.
    3. Papastamoulis, Panagiotis, 2018. "Overfitting Bayesian mixtures of factor analyzers with an unknown number of components," Computational Statistics & Data Analysis, Elsevier, vol. 124(C), pages 220-234.
    4. J. E. Mills & C. A. Field & D. J. Dupuis, 2002. "Marginally Specified Generalized Linear Mixed Models: A Robust Approach," Biometrics, The International Biometric Society, vol. 58(4), pages 727-734, December.
    5. Montanari, Angela & Viroli, Cinzia, 2011. "Maximum likelihood estimation of mixtures of factor analyzers," Computational Statistics & Data Analysis, Elsevier, vol. 55(9), pages 2712-2723, September.
    6. Jan Pablo Burgard & Patricia Dörr & Ralf Münnich, 2020. "Monte-Carlo Simulation Studies in Survey Statistics – An Appraisal," Research Papers in Economics 2020-04, University of Trier, Department of Economics.
    7. Lin, Tsung-I & McNicholas, Paul D. & Ho, Hsiu J., 2014. "Capturing patterns via parsimonious t mixture models," Statistics & Probability Letters, Elsevier, vol. 88(C), pages 80-87.
    8. Karl, Andrew T. & Yang, Yan & Lohr, Sharon L., 2014. "Computation of maximum likelihood estimates for multiresponse generalized linear mixed models with non-nested, correlated random effects," Computational Statistics & Data Analysis, Elsevier, vol. 73(C), pages 146-162.
    9. Leila Amiri & Mojtaba Khazaei & Mojtaba Ganjali, 2018. "A mixture latent variable model for modeling mixed data in heterogeneous populations and its applications," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 102(1), pages 95-115, January.
    10. Rosychuk, Rhonda J. & Shofiqul Islam, 2009. "Parameter estimation in a model for misclassified Markov data -- a Bayesian approach," Computational Statistics & Data Analysis, Elsevier, vol. 53(11), pages 3805-3816, September.
    11. Poncela, Pilar & Ruiz, Esther & Miranda, Karen, 2021. "Factor extraction using Kalman filter and smoothing: This is not just another survey," International Journal of Forecasting, Elsevier, vol. 37(4), pages 1399-1425.
    12. Savannah Wei Shi & Michel Wedel & F. G. M. (Rik) Pieters, 2013. "Information Acquisition During Online Decision Making: A Model-Based Exploration Using Eye-Tracking Data," Management Science, INFORMS, vol. 59(5), pages 1009-1026, May.
    13. Patricia Dörr & Jan Pablo Burgard, 2019. "Data-driven transformations and survey-weighting for linear mixed models," Research Papers in Economics 2019-16, University of Trier, Department of Economics.
    14. An, Xinming & Bentler, Peter M., 2012. "Efficient direct sampling MCEM algorithm for latent variable models with binary responses," Computational Statistics & Data Analysis, Elsevier, vol. 56(2), pages 231-244.
    15. Hongbin Zhang & Lang Wu, 2018. "A non‐linear model for censored and mismeasured time varying covariates in survival models, with applications in human immunodeficiency virus and acquired immune deficiency syndrome studies," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 67(5), pages 1437-1450, November.
    16. Tortora, Cristina & Franczak, Brian C. & Bagnato, Luca & Punzo, Antonio, 2024. "A Laplace-based model with flexible tail behavior," Computational Statistics & Data Analysis, Elsevier, vol. 192(C).
    17. Tingting Huang & Gilbert Saporta & Huiwen Wang & Shanshan Wang, 2021. "A robust spatial autoregressive scalar-on-function regression with t-distribution," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 15(1), pages 57-81, March.
    18. Angelo Moretti, 2023. "Estimation of small area proportions under a bivariate logistic mixed model," Quality & Quantity: International Journal of Methodology, Springer, vol. 57(4), pages 3663-3684, August.
    19. Charles Bouveyron & Camille Brunet-Saumard, 2014. "Discriminative variable selection for clustering with the sparse Fisher-EM algorithm," Computational Statistics, Springer, vol. 29(3), pages 489-513, June.
    20. Fan, Chunpeng & Zhang, Donghui, 2014. "Wald-type rank tests: A GEE approach," Computational Statistics & Data Analysis, Elsevier, vol. 74(C), pages 1-16.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zbw:caseps:200424. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ZBW - Leibniz Information Centre for Economics (email available below). General contact details of provider: https://edirc.repec.org/data/cahubde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.