IDEAS home Printed from https://ideas.repec.org/p/zbw/caseps/200424.html
   My bibliography  Save this paper

The EM Algorithm

Author

Listed:
  • McLachlan, Geoffrey J.
  • Krishnan, Thriyambakam
  • Ng, See Ket

Abstract

The Expectation-Maximization (EM) algorithm is a broadly applicable approach to the iterative computation of maximum likelihood (ML) estimates, useful in a variety of incomplete-data problems. Maximum likelihood estimation and likelihood-based inference are of central importance in statistical theory and data analysis. Maximum likelihood estimation is a general-purpose method with attractive properties. It is the most-often used estimation technique in the frequentist framework; it is also relevant in the Bayesian framework (Chapter III.11). Often Bayesian solutions are justified with the help of likelihoods and maximum likelihood estimates (MLE), and Bayesian solutions are similar to penalized likelihood estimates. Maximum likelihood estimation is an ubiquitous technique and is used extensively in every area where statistical techniques are used.

Suggested Citation

  • McLachlan, Geoffrey J. & Krishnan, Thriyambakam & Ng, See Ket, 2004. "The EM Algorithm," Papers 2004,24, Humboldt University of Berlin, Center for Applied Statistics and Economics (CASE).
    • Handle: RePEc:zbw:caseps:200424
    as

    Download full text from publisher

    File URL: https://www.econstor.eu/bitstream/10419/22198/1/24_tk_gm_skn.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. M. Jamshidian & R. I. Jennrich, 2000. "Standard errors for EM estimation," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 62(2), pages 257-270.
    2. J. G. Booth & J. P. Hobert, 1999. "Maximizing generalized linear mixed model likelihoods with an automated Monte Carlo EM algorithm," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 61(1), pages 265-285.
    3. McLachlan, G. J. & Peel, D. & Bean, R. W., 2003. "Modelling high-dimensional data by mixtures of factor analyzers," Computational Statistics & Data Analysis, Elsevier, vol. 41(3-4), pages 379-388, January.
    4. Robert, Christian P. & Celeux, Gilles & Diebolt, Jean, 1993. "Bayesian estimation of hidden Markov chains: a stochastic implementation," Statistics & Probability Letters, Elsevier, vol. 16(1), pages 77-83, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. James C. Spall, 2012. "Cyclic Seesaw Process for Optimization and Identification," Journal of Optimization Theory and Applications, Springer, vol. 154(1), pages 187-208, July.
    2. Ke-Hai Yuan & Peter Bentler, 2004. "On the asymptotic distributions of two statistics for two-level covariance structure models within the class of elliptical distributions," Psychometrika, Springer;The Psychometric Society, vol. 69(3), pages 437-457, September.
    3. Christophe Genolini & Bruno Falissard, 2010. "KmL: k-means for longitudinal data," Computational Statistics, Springer, vol. 25(2), pages 317-328, June.
    4. Feinerer, Ingo & Hornik, Kurt & Meyer, David, 2008. "Text Mining Infrastructure in R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 25(i05).
    5. Żyromski, Andrzej & Szulczewski, Wiesław & Biniak-Pieróg, Małgorzata & Jakubowski, Wojciech, 2016. "The estimation of basket willow (Salix viminalis) yield – New approach. Part I: Background and statistical description," Renewable and Sustainable Energy Reviews, Elsevier, vol. 65(C), pages 1118-1126.
    6. Guillaume Horny, 2009. "Inference in mixed proportional hazard models with K random effects," Statistical Papers, Springer, vol. 50(3), pages 481-499, June.
    7. Jakubowski, Wojciech & Szulczewski, Wiesław & Żyromski, Andrzej & Biniak-Pieróg, Małgorzata, 2016. "The estimation of basket willow (Salix viminalis) yield – New approach, Part II: Theoretical model and its practical application," Renewable and Sustainable Energy Reviews, Elsevier, vol. 66(C), pages 843-851.
    8. Qunqiang Feng & Hosam Mahmoud & Alois Panholzer, 2008. "Limit laws for the Randić index of random binary tree models," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 60(2), pages 319-343, June.
    9. Ke-Hai Yuan & Kentaro Hayashi, 2005. "On muthén’s maximum likelihood for two-level covariance structure models," Psychometrika, Springer;The Psychometric Society, vol. 70(1), pages 147-167, March.
    10. Ringle, Christian M., 2006. "Segmentation for path models and unobserved heterogeneity: The finite mixture partial least squares approach," MPRA Paper 10734, University Library of Munich, Germany.
    11. Orozco-Garcia, Carolina & Schmeiser, Hato, 2015. "How sensitive is the pricing of lookback and interest rate guarantees when changing the modelling assumptions?," Insurance: Mathematics and Economics, Elsevier, vol. 65(C), pages 77-93.
    12. Saeedeh Eskandari & Mahdis Amiri & Nitheshnirmal Sãdhasivam & Hamid Reza Pourghasemi, 2020. "Comparison of new individual and hybrid machine learning algorithms for modeling and mapping fire hazard: a supplementary analysis of fire hazard in different counties of Golestan Province in Iran," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 104(1), pages 305-327, October.
    13. Branislav Panić & Jernej Klemenc & Marko Nagode, 2020. "Improved Initialization of the EM Algorithm for Mixture Model Parameter Estimation," Mathematics, MDPI, vol. 8(3), pages 1-29, March.
    14. Fordellone, Mario & Vichi, Maurizio, 2020. "Finding groups in structural equation modeling through the partial least squares algorithm," Computational Statistics & Data Analysis, Elsevier, vol. 147(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Shu Yang & Jae Kwang Kim, 2016. "Likelihood-based Inference with Missing Data Under Missing-at-Random," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 43(2), pages 436-454, June.
    2. Hemant Kulkarni & Jayabrata Biswas & Kiranmoy Das, 2019. "A joint quantile regression model for multiple longitudinal outcomes," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 103(4), pages 453-473, December.
    3. Tatiyana V. Apanasovich & David Ruppert & Joanne R. Lupton & Natasa Popovic & Nancy D. Turner & Robert S. Chapkin & Raymond J. Carroll, 2008. "Aberrant Crypt Foci and Semiparametric Modeling of Correlated Binary Data," Biometrics, The International Biometric Society, vol. 64(2), pages 490-500, June.
    4. Matteo Barigozzi & Matteo Luciani, 2019. "Quasi Maximum Likelihood Estimation and Inference of Large Approximate Dynamic Factor Models via the EM algorithm," Papers 1910.03821, arXiv.org, revised Sep 2024.
    5. Ricardo Smith Ramírez, 2007. "FIML estimation of treatment effect models with endogenous selection and multiple censored responses via a Monte Carlo EM Algorithm," Working Papers DTE 403, CIDE, División de Economía.
    6. Wan-Lun Wang & Tsung-I Lin, 2013. "An efficient ECM algorithm for maximum likelihood estimation in mixtures of t-factor analyzers," Computational Statistics, Springer, vol. 28(2), pages 751-769, April.
    7. Brent A. Coull & Alan Agresti, 2000. "Random Effects Modeling of Multiple Binomial Responses Using the Multivariate Binomial Logit-Normal Distribution," Biometrics, The International Biometric Society, vol. 56(1), pages 73-80, March.
    8. Alessandro Casa & Andrea Cappozzo & Michael Fop, 2022. "Group-Wise Shrinkage Estimation in Penalized Model-Based Clustering," Journal of Classification, Springer;The Classification Society, vol. 39(3), pages 648-674, November.
    9. Papastamoulis, Panagiotis, 2018. "Overfitting Bayesian mixtures of factor analyzers with an unknown number of components," Computational Statistics & Data Analysis, Elsevier, vol. 124(C), pages 220-234.
    10. J. E. Mills & C. A. Field & D. J. Dupuis, 2002. "Marginally Specified Generalized Linear Mixed Models: A Robust Approach," Biometrics, The International Biometric Society, vol. 58(4), pages 727-734, December.
    11. Dirick, Lore & Claeskens, Gerda & Vasnev, Andrey & Baesens, Bart, 2022. "A hierarchical mixture cure model with unobserved heterogeneity for credit risk," Econometrics and Statistics, Elsevier, vol. 22(C), pages 39-55.
    12. repec:jss:jstsof:18:i06 is not listed on IDEAS
    13. Montanari, Angela & Viroli, Cinzia, 2011. "Maximum likelihood estimation of mixtures of factor analyzers," Computational Statistics & Data Analysis, Elsevier, vol. 55(9), pages 2712-2723, September.
    14. Jan Pablo Burgard & Patricia Dörr & Ralf Münnich, 2020. "Monte-Carlo Simulation Studies in Survey Statistics – An Appraisal," Research Papers in Economics 2020-04, University of Trier, Department of Economics.
    15. Lin, Tsung-I & McNicholas, Paul D. & Ho, Hsiu J., 2014. "Capturing patterns via parsimonious t mixture models," Statistics & Probability Letters, Elsevier, vol. 88(C), pages 80-87.
    16. Karl, Andrew T. & Yang, Yan & Lohr, Sharon L., 2014. "Computation of maximum likelihood estimates for multiresponse generalized linear mixed models with non-nested, correlated random effects," Computational Statistics & Data Analysis, Elsevier, vol. 73(C), pages 146-162.
    17. Peter Halpin & Paul Boeck, 2013. "Modelling Dyadic Interaction with Hawkes Processes," Psychometrika, Springer;The Psychometric Society, vol. 78(4), pages 793-814, October.
    18. Leila Amiri & Mojtaba Khazaei & Mojtaba Ganjali, 2018. "A mixture latent variable model for modeling mixed data in heterogeneous populations and its applications," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 102(1), pages 95-115, January.
    19. Kauermann, Goran & Xu, Ronghui & Vaida, Florin, 2008. "Stacked Laplace-EM algorithm for duration models with time-varying and random effects," Computational Statistics & Data Analysis, Elsevier, vol. 52(5), pages 2514-2528, January.
    20. Rosychuk, Rhonda J. & Shofiqul Islam, 2009. "Parameter estimation in a model for misclassified Markov data -- a Bayesian approach," Computational Statistics & Data Analysis, Elsevier, vol. 53(11), pages 3805-3816, September.
    21. Steffen Nestler & Sarah Humberg, 2022. "A Lasso and a Regression Tree Mixed-Effect Model with Random Effects for the Level, the Residual Variance, and the Autocorrelation," Psychometrika, Springer;The Psychometric Society, vol. 87(2), pages 506-532, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zbw:caseps:200424. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ZBW - Leibniz Information Centre for Economics (email available below). General contact details of provider: https://edirc.repec.org/data/cahubde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.