IDEAS home Printed from https://ideas.repec.org/a/bla/jorssb/v59y1997i3p569-587.html
   My bibliography  Save this article

Acceleration of the EM Algorithm by using Quasi‐Newton Methods

Author

Listed:
  • Mortaza Jamshidian
  • Robert I. Jennrich

Abstract

The EM algorithm is a popular method for maximum likelihood estimation. Its simplicity in many applications and desirable convergence properties make it very attractive. Its sometimes slow convergence, however, has prompted researchers to propose methods to accelerate it. We review these methods, classifying them into three groups: pure, hybrid and EM‐type accelerators. We propose a new pure and a new hybrid accelerator both based on quasi‐Newton methods and numerically compare these and two other quasi‐Newton accelerators. For this we use examples in each of three areas: Poisson mixtures, the estimation of covariance from incomplete data and multivariate normal mixtures. In these comparisons, the new hybrid accelerator was fastest on most of the examples and often dramatically so. In some cases it accelerated the EM algorithm by factors of over 100. The new pure accelerator is very simple to implement and competed well with the other accelerators. It accelerated the EM algorithm in some cases by factors of over 50. To obtain standard errors, we propose to approximate the inverse of the observed information matrix by using auxiliary output from the new hybrid accelerator. A numerical evaluation of these approximations indicates that they may be useful at least for exploratory purposes.

Suggested Citation

  • Mortaza Jamshidian & Robert I. Jennrich, 1997. "Acceleration of the EM Algorithm by using Quasi‐Newton Methods," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 59(3), pages 569-587.
    • Handle: RePEc:bla:jorssb:v:59:y:1997:i:3:p:569-587
    DOI: 10.1111/1467-9868.00083
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/1467-9868.00083
    Download Restriction: no
    File URL: https://libkey.io/10.1111/1467-9868.00083?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Fung, Tsz Chai, 2022. "Maximum weighted likelihood estimator for robust heavy-tail modelling of finite mixture models," Insurance: Mathematics and Economics, Elsevier, vol. 107(C), pages 180-198.
    2. Zhou, Lin & Tang, Yayong, 2021. "Linearly preconditioned nonlinear conjugate gradient acceleration of the PX-EM algorithm," Computational Statistics & Data Analysis, Elsevier, vol. 155(C).
    3. Iain L. MacDonald, 2021. "Is EM really necessary here? Examples where it seems simpler not to use EM," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 105(4), pages 629-647, December.
    4. Takeshi Fukasawa, 2024. "Fast and simple inner-loop algorithms of static / dynamic BLP estimations," Papers 2404.04494, arXiv.org, revised Oct 2024.
    5. Kaarina Matilainen & Esa A Mäntysaari & Martin H Lidauer & Ismo Strandén & Robin Thompson, 2013. "Employing a Monte Carlo Algorithm in Newton-Type Methods for Restricted Maximum Likelihood Estimation of Genetic Parameters," PLOS ONE, Public Library of Science, vol. 8(12), pages 1-7, December.
    6. Jurgen A. Doornik, 2018. "Accelerated Estimation of Switching Algorithms: The Cointegrated VAR Model and Other Applications," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 45(2), pages 283-300, June.
    7. Rasool Roozegar & G. G. Hamedani & Leila Amiri & Fatemeh Esfandiyari, 2020. "A New Family of Lifetime Distributions: Theory, Application and Characterizations," Annals of Data Science, Springer, vol. 7(1), pages 109-138, March.
    8. Saâdaoui, Foued, 2023. "Randomized extrapolation for accelerating EM-type fixed-point algorithms," Journal of Multivariate Analysis, Elsevier, vol. 196(C).
    9. Peter Arcidiacono & Robert A. Miller, 2011. "Conditional Choice Probability Estimation of Dynamic Discrete Choice Models With Unobserved Heterogeneity," Econometrica, Econometric Society, vol. 79(6), pages 1823-1867, November.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssb:v:59:y:1997:i:3:p:569-587. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/rssssea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.