IDEAS home Printed from https://ideas.repec.org/a/eee/jmvana/v114y2013icp127-160.html
   My bibliography  Save this article

Sparse principal component analysis by choice of norm

Author

Listed:
  • Qi, Xin
  • Luo, Ruiyan
  • Zhao, Hongyu

Abstract

Recent years have seen the developments of several methods for sparse principal component analysis due to its importance in the analysis of high dimensional data. Despite the demonstration of their usefulness in practical applications, they are limited in terms of lack of orthogonality in the loadings (coefficients) of different principal components, the existence of correlation in the principal components, the expensive computation needed, and the lack of theoretical results such as consistency in high-dimensional situations. In this paper, we propose a new sparse principal component analysis method by introducing a new norm to replace the usual norm in traditional eigenvalue problems, and propose an efficient iterative algorithm to solve the optimization problems. With this method, we can efficiently obtain uncorrelated principal components or orthogonal loadings, and achieve the goal of explaining a high percentage of variations with sparse linear combinations. Due to the strict convexity of the new norm, we can prove the convergence of the iterative method and provide the detailed characterization of the limits. We also prove that the obtained principal component is consistent for a single component model in high dimensional situations. As illustration, we apply this method to real gene expression data with competitive results.

Suggested Citation

  • Qi, Xin & Luo, Ruiyan & Zhao, Hongyu, 2013. "Sparse principal component analysis by choice of norm," Journal of Multivariate Analysis, Elsevier, vol. 114(C), pages 127-160.
    • Handle: RePEc:eee:jmvana:v:114:y:2013:i:c:p:127-160
    DOI: 10.1016/j.jmva.2012.07.004
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0047259X12001728
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jmva.2012.07.004?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Johnstone, Iain M. & Lu, Arthur Yu, 2009. "On Consistency and Sparsity for Principal Components Analysis in High Dimensions," Journal of the American Statistical Association, American Statistical Association, vol. 104(486), pages 682-693.
    2. Shen, Haipeng & Huang, Jianhua Z., 2008. "Sparse principal component analysis via regularized low rank matrix approximation," Journal of Multivariate Analysis, Elsevier, vol. 99(6), pages 1015-1034, July.
    3. Trendafilov, Nickolay T. & Jolliffe, Ian T., 2006. "Projected gradient approach to the numerical solution of the SCoTLASS," Computational Statistics & Data Analysis, Elsevier, vol. 50(1), pages 242-253, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Carrizosa, Emilio & Guerrero, Vanesa, 2014. "Biobjective sparse principal component analysis," Journal of Multivariate Analysis, Elsevier, vol. 132(C), pages 151-159.
    2. Roberto Casarin & Fausto Corradin & Francesco Ravazzolo & Nguyen Domenico Sartore & Wing-Keung Wong, 2020. "A Scoring Rule for Factor and Autoregressive Models Under Misspecification," Advances in Decision Sciences, Asia University, Taiwan, vol. 24(2), pages 66-103, June.
    3. D. F. Nwosu & V. U. Ekhosuehi & J. I. Mbegbu, 2020. "Performance of Some Factor Analysis Techniques," Annals of Data Science, Springer, vol. 7(2), pages 209-242, June.
    4. Luo, Ruiyan & Qi, Xin, 2017. "Signal extraction approach for sparse multivariate response regression," Journal of Multivariate Analysis, Elsevier, vol. 153(C), pages 83-97.
    5. Nickolay Trendafilov, 2014. "From simple structure to sparse components: a review," Computational Statistics, Springer, vol. 29(3), pages 431-454, June.
    6. Mitzi Cubilla-Montilla & Ana Belén Nieto-Librero & M. Purificación Galindo-Villardón & Carlos A. Torres-Cubilla, 2021. "Sparse HJ Biplot: A New Methodology via Elastic Net," Mathematics, MDPI, vol. 9(11), pages 1-15, June.
    7. Xin Qi & Ruiyan Luo, 2015. "Sparse Principal Component Analysis in Hilbert Space," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 42(1), pages 270-289, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Xin Qi & Ruiyan Luo, 2015. "Sparse Principal Component Analysis in Hilbert Space," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 42(1), pages 270-289, March.
    2. Nickolay Trendafilov, 2014. "From simple structure to sparse components: a review," Computational Statistics, Springer, vol. 29(3), pages 431-454, June.
    3. Rosember Guerra-Urzola & Katrijn Van Deun & Juan C. Vera & Klaas Sijtsma, 2021. "A Guide for Sparse PCA: Model Comparison and Applications," Psychometrika, Springer;The Psychometric Society, vol. 86(4), pages 893-919, December.
    4. Jianqing Fan & Yuan Liao & Martina Mincheva, 2013. "Large covariance estimation by thresholding principal orthogonal complements," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 75(4), pages 603-680, September.
    5. Lê Cao Kim-Anh & Rossouw Debra & Robert-Granié Christèle & Besse Philippe, 2008. "A Sparse PLS for Variable Selection when Integrating Omics Data," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 7(1), pages 1-32, November.
    6. Guerra Urzola, Rosember & Van Deun, Katrijn & Vera, J. C. & Sijtsma, K., 2021. "A guide for sparse PCA : Model comparison and applications," Other publications TiSEM 4d35b931-7f49-444b-b92f-a, Tilburg University, School of Economics and Management.
    7. Lei Wang & Xin Liu & Yin Zhang, 2023. "A communication-efficient and privacy-aware distributed algorithm for sparse PCA," Computational Optimization and Applications, Springer, vol. 85(3), pages 1033-1072, July.
    8. Shen, Dan & Shen, Haipeng & Marron, J.S., 2013. "Consistency of sparse PCA in High Dimension, Low Sample Size contexts," Journal of Multivariate Analysis, Elsevier, vol. 115(C), pages 317-333.
    9. Peter Bentler & Jan Leeuw, 2011. "Factor Analysis via Components Analysis," Psychometrika, Springer;The Psychometric Society, vol. 76(3), pages 461-470, July.
    10. Ningning Xia & Zhidong Bai, 2019. "Convergence rate of eigenvector empirical spectral distribution of large Wigner matrices," Statistical Papers, Springer, vol. 60(3), pages 983-1015, June.
    11. Fang, Kuangnan & Fan, Xinyan & Zhang, Qingzhao & Ma, Shuangge, 2018. "Integrative sparse principal component analysis," Journal of Multivariate Analysis, Elsevier, vol. 166(C), pages 1-16.
    12. Candelon, B. & Hurlin, C. & Tokpavi, S., 2012. "Sampling error and double shrinkage estimation of minimum variance portfolios," Journal of Empirical Finance, Elsevier, vol. 19(4), pages 511-527.
    13. Yixuan Qiu & Jing Lei & Kathryn Roeder, 2023. "Gradient-based sparse principal component analysis with extensions to online learning," Biometrika, Biometrika Trust, vol. 110(2), pages 339-360.
    14. Yata, Kazuyoshi & Aoshima, Makoto, 2013. "PCA consistency for the power spiked model in high-dimensional settings," Journal of Multivariate Analysis, Elsevier, vol. 122(C), pages 334-354.
    15. Asai, Manabu & McAleer, Michael, 2015. "Forecasting co-volatilities via factor models with asymmetry and long memory in realized covariance," Journal of Econometrics, Elsevier, vol. 189(2), pages 251-262.
    16. Jushan Bai & Serena Ng, 2020. "Simpler Proofs for Approximate Factor Models of Large Dimensions," Papers 2008.00254, arXiv.org.
    17. Thomas Despois & Catherine Doz, 2023. "Identifying and interpreting the factors in factor models via sparsity: Different approaches," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 38(4), pages 533-555, June.
    18. Wang, Shao-Hsuan & Huang, Su-Yun, 2022. "Perturbation theory for cross data matrix-based PCA," Journal of Multivariate Analysis, Elsevier, vol. 190(C).
    19. Silin, Igor & Spokoiny, Vladimir, 2018. "Bayesian inference for spectral projectors of covariance matrix," IRTG 1792 Discussion Papers 2018-027, Humboldt University of Berlin, International Research Training Group 1792 "High Dimensional Nonstationary Time Series".
    20. Barigozzi, Matteo & Trapani, Lorenzo, 2020. "Sequential testing for structural stability in approximate factor models," Stochastic Processes and their Applications, Elsevier, vol. 130(8), pages 5149-5187.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jmvana:v:114:y:2013:i:c:p:127-160. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/622892/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.