IDEAS home Printed from https://ideas.repec.org/a/spr/compst/v29y2014i3p431-454.html
   My bibliography  Save this article

From simple structure to sparse components: a review

Author

Listed:
  • Nickolay Trendafilov

Abstract

The article begins with a review of the main approaches for interpretation the results from principal component analysis (PCA) during the last 50–60 years. The simple structure approach is compared to the modern approach of sparse PCA where interpretable solutions are directly obtained. It is shown that their goals are identical but they differ by the way they are realized. Next, the most popular and influential methods for sparse PCA are briefly reviewed. In the remaining part of the paper, a new approach to define sparse PCA is introduced. Several alternative definitions are considered and illustrated on a well-known data set. Finally, it is demonstrated, how one of these possible versions of sparse PCA can be used as a sparse alternative to the classical rotation methods. Copyright Springer-Verlag Berlin Heidelberg 2014

Suggested Citation

  • Nickolay Trendafilov, 2014. "From simple structure to sparse components: a review," Computational Statistics, Springer, vol. 29(3), pages 431-454, June.
    • Handle: RePEc:spr:compst:v:29:y:2014:i:3:p:431-454
    DOI: 10.1007/s00180-013-0434-5
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s00180-013-0434-5
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s00180-013-0434-5?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Johnstone, Iain M. & Lu, Arthur Yu, 2009. "On Consistency and Sparsity for Principal Components Analysis in High Dimensions," Journal of the American Statistical Association, American Statistical Association, vol. 104(486), pages 682-693.
    2. Shen, Haipeng & Huang, Jianhua Z., 2008. "Sparse principal component analysis via regularized low rank matrix approximation," Journal of Multivariate Analysis, Elsevier, vol. 99(6), pages 1015-1034, July.
    3. Doyo Enki & Nickolay Trendafilov & Ian Jolliffe, 2013. "A clustering approach to interpretable principal components," Journal of Applied Statistics, Taylor & Francis Journals, vol. 40(3), pages 583-599.
    4. Doyo Enki & Nickolay Trendafilov, 2012. "Sparse principal components by semi-partition clustering," Computational Statistics, Springer, vol. 27(4), pages 605-626, December.
    5. S. K. Vines, 2000. "Simple principal components," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 49(4), pages 441-451.
    6. Hugh Chipman & Hong Gu, 2005. "Interpretable dimension reduction," Journal of Applied Statistics, Taylor & Francis Journals, vol. 32(9), pages 969-987.
    7. Vichi, Maurizio & Saporta, Gilbert, 2009. "Clustering and disjoint principal component analysis," Computational Statistics & Data Analysis, Elsevier, vol. 53(8), pages 3194-3208, June.
    8. Valentin Rousson & Theo Gasser, 2004. "Simple component analysis," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 53(4), pages 539-555, November.
    9. Qi, Xin & Luo, Ruiyan & Zhao, Hongyu, 2013. "Sparse principal component analysis by choice of norm," Journal of Multivariate Analysis, Elsevier, vol. 114(C), pages 127-160.
    10. Carl Eckart & Gale Young, 1936. "The approximation of one matrix by another of lower rank," Psychometrika, Springer;The Psychometric Society, vol. 1(3), pages 211-218, September.
    11. repec:ucp:bkecon:9780226316529 is not listed on IDEAS
    12. J. N. R. Jeffers, 1967. "Two Case Studies in the Application of Principal Component Analysis," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 16(3), pages 225-236, November.
    13. Trendafilov, Nickolay T. & Jolliffe, Ian T., 2006. "Projected gradient approach to the numerical solution of the SCoTLASS," Computational Statistics & Data Analysis, Elsevier, vol. 50(1), pages 242-253, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jin, Shaobo & Moustaki, Irini & Yang-Wallentin, Fan, 2018. "Approximated penalized maximum likelihood for exploratory factor analysis: an orthogonal case," LSE Research Online Documents on Economics 88118, London School of Economics and Political Science, LSE Library.
    2. Shaobo Jin & Irini Moustaki & Fan Yang-Wallentin, 2018. "Approximated Penalized Maximum Likelihood for Exploratory Factor Analysis: An Orthogonal Case," Psychometrika, Springer;The Psychometric Society, vol. 83(3), pages 628-649, September.
    3. Thomas Despois & Catherine Doz, 2021. "Identifying and interpreting the factors in factor models via sparsity: Different approaches," Working Papers halshs-02235543, HAL.
    4. Ikemoto, Hiroki & Adachi, Kohei, 2016. "Sparse Tucker2 analysis of three-way data subject to a constrained number of zero elements in a core array," Computational Statistics & Data Analysis, Elsevier, vol. 98(C), pages 1-18.
    5. Nickolay T. Trendafilov & Sara Fontanella & Kohei Adachi, 2017. "Sparse Exploratory Factor Analysis," Psychometrika, Springer;The Psychometric Society, vol. 82(3), pages 778-794, September.
    6. Nerea González-García & Ana Belén Nieto-Librero & Purificación Galindo-Villardón, 2023. "CenetBiplot: a new proposal of sparse and orthogonal biplots methods by means of elastic net CSVD," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 17(1), pages 5-19, March.
    7. Liu, Xinyi Lin & Wallin, Gabriel & Chen, Yunxiao & Moustaki, Irini, 2023. "Rotation to sparse loadings using Lp losses and related inference problems," LSE Research Online Documents on Economics 118349, London School of Economics and Political Science, LSE Library.
    8. Kohei Adachi & Nickolay T. Trendafilov, 2016. "Sparse principal component analysis subject to prespecified cardinality of loadings," Computational Statistics, Springer, vol. 31(4), pages 1403-1427, December.
    9. Nickolay T. Trendafilov & Tsegay Gebrehiwot Gebru, 2016. "Recipes for sparse LDA of horizontal data," METRON, Springer;Sapienza Università di Roma, vol. 74(2), pages 207-221, August.
    10. Nickolay Trendafilov & Martin Kleinsteuber & Hui Zou, 2014. "Sparse matrices in data analysis," Computational Statistics, Springer, vol. 29(3), pages 403-405, June.
    11. Xinyi Liu & Gabriel Wallin & Yunxiao Chen & Irini Moustaki, 2023. "Rotation to Sparse Loadings Using $$L^p$$ L p Losses and Related Inference Problems," Psychometrika, Springer;The Psychometric Society, vol. 88(2), pages 527-553, June.
    12. Mitzi Cubilla-Montilla & Ana Belén Nieto-Librero & M. Purificación Galindo-Villardón & Carlos A. Torres-Cubilla, 2021. "Sparse HJ Biplot: A New Methodology via Elastic Net," Mathematics, MDPI, vol. 9(11), pages 1-15, June.
    13. Thomas Despois & Catherine Doz, 2022. "Identifying and interpreting the factors in factor models via sparsity : Different approaches," Working Papers halshs-03626503, HAL.
    14. Rosember Guerra-Urzola & Niek C. Schipper & Anya Tonne & Klaas Sijtsma & Juan C. Vera & Katrijn Deun, 2023. "Sparsifying the least-squares approach to PCA: comparison of lasso and cardinality constraint," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 17(1), pages 269-286, March.
    15. Guerra Urzola, Rosember & Van Deun, Katrijn & Vera, J. C. & Sijtsma, K., 2021. "A guide for sparse PCA : Model comparison and applications," Other publications TiSEM 4d35b931-7f49-444b-b92f-a, Tilburg University, School of Economics and Management.
    16. Rosember Guerra-Urzola & Katrijn Van Deun & Juan C. Vera & Klaas Sijtsma, 2021. "A Guide for Sparse PCA: Model Comparison and Applications," Psychometrika, Springer;The Psychometric Society, vol. 86(4), pages 893-919, December.
    17. Thomas Despois & Catherine Doz, 2022. "Identifying and interpreting the factors in factor models via sparsity : Different approaches," PSE Working Papers halshs-03626503, HAL.
    18. Thomas Despois & Catherine Doz, 2023. "Identifying and interpreting the factors in factor models via sparsity: Different approaches," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 38(4), pages 533-555, June.
    19. Kohei Adachi & Nickolay T. Trendafilov, 2018. "Sparsest factor analysis for clustering variables: a matrix decomposition approach," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 12(3), pages 559-585, September.
    20. Thomas Despois & Catherine Doz, 2021. "Identifying and interpreting the factors in factor models via sparsity: Different approaches," PSE Working Papers halshs-02235543, HAL.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kohei Adachi & Nickolay T. Trendafilov, 2016. "Sparse principal component analysis subject to prespecified cardinality of loadings," Computational Statistics, Springer, vol. 31(4), pages 1403-1427, December.
    2. Xin Qi & Ruiyan Luo, 2015. "Sparse Principal Component Analysis in Hilbert Space," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 42(1), pages 270-289, March.
    3. Trendafilov, Nickolay T. & Vines, Karen, 2009. "Simple and interpretable discrimination," Computational Statistics & Data Analysis, Elsevier, vol. 53(4), pages 979-989, February.
    4. Mitzi Cubilla-Montilla & Ana Belén Nieto-Librero & M. Purificación Galindo-Villardón & Carlos A. Torres-Cubilla, 2021. "Sparse HJ Biplot: A New Methodology via Elastic Net," Mathematics, MDPI, vol. 9(11), pages 1-15, June.
    5. Carrizosa, Emilio & Guerrero, Vanesa, 2014. "Biobjective sparse principal component analysis," Journal of Multivariate Analysis, Elsevier, vol. 132(C), pages 151-159.
    6. Nerea González-García & Ana Belén Nieto-Librero & Purificación Galindo-Villardón, 2023. "CenetBiplot: a new proposal of sparse and orthogonal biplots methods by means of elastic net CSVD," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 17(1), pages 5-19, March.
    7. Shen, Dan & Shen, Haipeng & Marron, J.S., 2013. "Consistency of sparse PCA in High Dimension, Low Sample Size contexts," Journal of Multivariate Analysis, Elsevier, vol. 115(C), pages 317-333.
    8. Sabatier, Robert & Reynès, Christelle, 2008. "Extensions of simple component analysis and simple linear discriminant analysis using genetic algorithms," Computational Statistics & Data Analysis, Elsevier, vol. 52(10), pages 4779-4789, June.
    9. Rosember Guerra-Urzola & Katrijn Van Deun & Juan C. Vera & Klaas Sijtsma, 2021. "A Guide for Sparse PCA: Model Comparison and Applications," Psychometrika, Springer;The Psychometric Society, vol. 86(4), pages 893-919, December.
    10. Jolliffe, Ian, 2022. "A 50-year personal journey through time with principal component analysis," Journal of Multivariate Analysis, Elsevier, vol. 188(C).
    11. T. F. Cox & D. S. Arnold, 2018. "Simple components," Journal of Applied Statistics, Taylor & Francis Journals, vol. 45(1), pages 83-99, January.
    12. Qi, Xin & Luo, Ruiyan & Zhao, Hongyu, 2013. "Sparse principal component analysis by choice of norm," Journal of Multivariate Analysis, Elsevier, vol. 114(C), pages 127-160.
    13. Guerra Urzola, Rosember & Van Deun, Katrijn & Vera, J. C. & Sijtsma, K., 2021. "A guide for sparse PCA : Model comparison and applications," Other publications TiSEM 4d35b931-7f49-444b-b92f-a, Tilburg University, School of Economics and Management.
    14. Adelaide Freitas & Eloísa Macedo & Maurizio Vichi, 2021. "An empirical comparison of two approaches for CDPCA in high-dimensional data," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 30(3), pages 1007-1031, September.
    15. Jushan Bai & Serena Ng, 2020. "Simpler Proofs for Approximate Factor Models of Large Dimensions," Papers 2008.00254, arXiv.org.
    16. Jin-Xing Liu & Yong Xu & Chun-Hou Zheng & Yi Wang & Jing-Yu Yang, 2012. "Characteristic Gene Selection via Weighting Principal Components by Singular Values," PLOS ONE, Public Library of Science, vol. 7(7), pages 1-10, July.
    17. Merola, Giovanni Maria & Chen, Gemai, 2019. "Projection sparse principal component analysis: An efficient least squares method," Journal of Multivariate Analysis, Elsevier, vol. 173(C), pages 366-382.
    18. Maurizio Vichi, 2017. "Disjoint factor analysis with cross-loadings," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 11(3), pages 563-591, September.
    19. Lei Wang & Xin Liu & Yin Zhang, 2023. "A communication-efficient and privacy-aware distributed algorithm for sparse PCA," Computational Optimization and Applications, Springer, vol. 85(3), pages 1033-1072, July.
    20. Mihee Lee & Haipeng Shen & Jianhua Z. Huang & J. S. Marron, 2010. "Biclustering via Sparse Singular Value Decomposition," Biometrics, The International Biometric Society, vol. 66(4), pages 1087-1095, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:compst:v:29:y:2014:i:3:p:431-454. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.