IDEAS home Printed from https://ideas.repec.org/a/spr/psycho/v81y2016i3d10.1007_s11336-015-9486-5.html
   My bibliography  Save this article

Factor analysis models via I-divergence optimization

Author

Listed:
  • Lorenzo Finesso

    (IEIIT - CNR)

  • Peter Spreij

    (Universiteit van Amsterdam)

Abstract

Given a positive definite covariance matrix $$\widehat{\Sigma }$$ Σ ^ of dimension n, we approximate it with a covariance of the form $$HH^\top +D$$ H H ⊤ + D , where H has a prescribed number $$k 0$$ D > 0 is diagonal. The quality of the approximation is gauged by the I-divergence between the zero mean normal laws with covariances $$\widehat{\Sigma }$$ Σ ^ and $$HH^\top +D$$ H H ⊤ + D , respectively. To determine a pair (H, D) that minimizes the I-divergence we construct, by lifting the minimization into a larger space, an iterative alternating minimization algorithm (AML) à la Csiszár–Tusnády. As it turns out, the proper choice of the enlarged space is crucial for optimization. The convergence of the algorithm is studied, with special attention given to the case where D is singular. The theoretical properties of the AML are compared to those of the popular EM algorithm for exploratory factor analysis. Inspired by the ECME (a Newton–Raphson variation on EM), we develop a similar variant of AML, called ACML, and in a few numerical experiments, we compare the performances of the four algorithms.

Suggested Citation

  • Lorenzo Finesso & Peter Spreij, 2016. "Factor analysis models via I-divergence optimization," Psychometrika, Springer;The Psychometric Society, vol. 81(3), pages 702-726, September.
    • Handle: RePEc:spr:psycho:v:81:y:2016:i:3:d:10.1007_s11336-015-9486-5
    DOI: 10.1007/s11336-015-9486-5
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11336-015-9486-5
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11336-015-9486-5?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Kohei Adachi, 2013. "Factor Analysis with EM Algorithm Never Gives Improper Solutions when Sample Covariance and Initial Parameter Matrices Are Proper," Psychometrika, Springer;The Psychometric Society, vol. 78(2), pages 380-394, April.
    2. C. Rao, 1955. "Estimation and tests of significance in factor analysis," Psychometrika, Springer;The Psychometric Society, vol. 20(2), pages 93-111, June.
    3. Walter Ledermann, 1937. "On the rank of the reduced correlational matrix in multiple-factor analysis," Psychometrika, Springer;The Psychometric Society, vol. 2(2), pages 85-93, June.
    4. Donald Rubin & Dorothy Thayer, 1982. "EM algorithms for ML factor analysis," Psychometrika, Springer;The Psychometric Society, vol. 47(1), pages 69-76, March.
    5. Zhao, Jianhua & Shi, Lei, 2014. "Automated learning of factor analysis with complete and incomplete data," Computational Statistics & Data Analysis, Elsevier, vol. 72(C), pages 205-218.
    6. Robert Jennrich & Stephen Robinson, 1969. "A Newton-Raphson algorithm for maximum likelihood factor analysis," Psychometrika, Springer;The Psychometric Society, vol. 34(1), pages 111-123, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sik-Yum Lee, 1980. "Estimation of covariance structure models with parameters subject to functional restraints," Psychometrika, Springer;The Psychometric Society, vol. 45(3), pages 309-324, September.
    2. Bai, Jushan, 2024. "Likelihood approach to dynamic panel models with interactive effects," Journal of Econometrics, Elsevier, vol. 240(1).
    3. Yutaka Kano, 1990. "Noniterative estimation and the choice of the number of factors in exploratory factor analysis," Psychometrika, Springer;The Psychometric Society, vol. 55(2), pages 277-291, June.
    4. Nikolaos Zirogiannis & Yorghos Tripodis, 2018. "Dynamic factor analysis for short panels: estimating performance trajectories for water utilities," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 27(1), pages 131-150, March.
    5. Kohei Adachi, 2013. "Factor Analysis with EM Algorithm Never Gives Improper Solutions when Sample Covariance and Initial Parameter Matrices Are Proper," Psychometrika, Springer;The Psychometric Society, vol. 78(2), pages 380-394, April.
    6. Kohei Adachi, 2022. "Factor Analysis Procedures Revisited from the Comprehensive Model with Unique Factors Decomposed into Specific Factors and Errors," Psychometrika, Springer;The Psychometric Society, vol. 87(3), pages 967-991, September.
    7. Wan-Lun Wang & Tsung-I Lin, 2020. "Automated learning of mixtures of factor analysis models with missing information," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 29(4), pages 1098-1124, December.
    8. Wang, Wan-Lun & Castro, Luis M. & Lin, Tsung-I, 2017. "Automated learning of t factor analysis models with complete and incomplete data," Journal of Multivariate Analysis, Elsevier, vol. 161(C), pages 157-171.
    9. Ihara, Masamori & Kano, Yutaka, 1995. "Identifiability of full, marginal, and conditional factor analysis models," Statistics & Probability Letters, Elsevier, vol. 23(4), pages 343-350, June.
    10. Benjamin Williams, 2018. "Identification of the Linear Factor Model," Working Papers 2018-002, The George Washington University, Department of Economics, H. O. Stekler Research Program on Forecasting.
    11. Zirogiannis, Nikolaos & Tripodis, Yorghos, 2013. "A Generalized Dynamic Factor Model for Panel Data: Estimation with a Two-Cycle Conditional Expectation-Maximization Algorithm," Working Paper Series 142752, University of Massachusetts, Amherst, Department of Resource Economics.
    12. Dorota Toczydlowska & Gareth W. Peters & Man Chung Fung & Pavel V. Shevchenko, 2017. "Stochastic Period and Cohort Effect State-Space Mortality Models Incorporating Demographic Factors via Probabilistic Robust Principal Components," Risks, MDPI, vol. 5(3), pages 1-77, July.
    13. Aßmann, Christian & Boysen-Hogrefe, Jens & Pape, Markus, 2012. "The directional identification problem in Bayesian factor analysis: An ex-post approach," Kiel Working Papers 1799, Kiel Institute for the World Economy (IfW Kiel).
    14. Kinkyo, Takuji, 2021. "Region-wide connectedness of Asian equity and currency markets," The North American Journal of Economics and Finance, Elsevier, vol. 58(C).
    15. Chen, Derek H. C. & Gawande, Kishore, 2007. "Underlying dimensions of knowledge assessment : factor analysis of the knowledge assessment methodology data," Policy Research Working Paper Series 4216, The World Bank.
    16. Conti, Gabriella & Frühwirth-Schnatter, Sylvia & Heckman, James J. & Piatek, Rémi, 2014. "Bayesian exploratory factor analysis," Journal of Econometrics, Elsevier, vol. 183(1), pages 31-57.
    17. Papastamoulis, Panagiotis, 2018. "Overfitting Bayesian mixtures of factor analyzers with an unknown number of components," Computational Statistics & Data Analysis, Elsevier, vol. 124(C), pages 220-234.
    18. Zhou, Lin & Tang, Yayong, 2021. "Linearly preconditioned nonlinear conjugate gradient acceleration of the PX-EM algorithm," Computational Statistics & Data Analysis, Elsevier, vol. 155(C).
    19. Kim, Jiwhan & Nam, Changi & Ryu, Min Ho, 2020. "IPTV vs. emerging video services: Dilemma of telcos to upgrade the broadband," Telecommunications Policy, Elsevier, vol. 44(4).
    20. Mao Takongmo, Charles Olivier & Stevanovic, Dalibor, 2015. "Selection Of The Number Of Factors In Presence Of Structural Instability: A Monte Carlo Study," L'Actualité Economique, Société Canadienne de Science Economique, vol. 91(1-2), pages 177-233, Mars-Juin.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:psycho:v:81:y:2016:i:3:d:10.1007_s11336-015-9486-5. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.