IDEAS home Printed from https://ideas.repec.org/a/spr/advdac/v8y2014i2p217-226.html
   My bibliography  Save this article

Estimating common principal components in high dimensions

Author

Listed:
  • Ryan Browne
  • Paul McNicholas

Abstract

We consider the problem of minimizing an objective function that depends on an orthonormal matrix. This situation is encountered, for example, when looking for common principal components. The Flury method is a popular approach but is not effective for higher dimensional problems. We obtain several simple majorization–minimization (MM) algorithms that provide solutions to this problem and are effective in higher dimensions. We use mixture model-based clustering applications to illustrate our MM algorithms. We then use simulated data to compare them with other approaches, with comparisons drawn with respect to convergence and computational time. Copyright Springer-Verlag Berlin Heidelberg 2014

Suggested Citation

  • Ryan Browne & Paul McNicholas, 2014. "Estimating common principal components in high dimensions," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 8(2), pages 217-226, June.
    • Handle: RePEc:spr:advdac:v:8:y:2014:i:2:p:217-226
    DOI: 10.1007/s11634-013-0139-1
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s11634-013-0139-1
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s11634-013-0139-1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Schott, James R., 1998. "Estimating correlation matrices that have common eigenvectors," Computational Statistics & Data Analysis, Elsevier, vol. 27(4), pages 445-459, June.
    2. Bouveyron, C. & Girard, S. & Schmid, C., 2007. "High-dimensional data clustering," Computational Statistics & Data Analysis, Elsevier, vol. 52(1), pages 502-519, September.
    3. W. Krzanowski, 1990. "Between-group analysis with heterogeneous covariance matrices: The common principal component model," Journal of Classification, Springer;The Classification Society, vol. 7(1), pages 81-98, March.
    4. Biernacki, Christophe & Celeux, Gilles & Govaert, Gerard & Langrognet, Florent, 2006. "Model-based cluster and discriminant analysis with the MIXMOD software," Computational Statistics & Data Analysis, Elsevier, vol. 51(2), pages 587-600, November.
    5. Merbouha, A. & Mkhadri, A., 2004. "Regularization of the location model in discrimination with mixed discrete and continuous variables," Computational Statistics & Data Analysis, Elsevier, vol. 45(3), pages 563-576, April.
    6. Kiers, Henk A. L., 2002. "Setting up alternating least squares and iterative majorization algorithms for solving various matrix optimization problems," Computational Statistics & Data Analysis, Elsevier, vol. 41(1), pages 157-170, November.
    7. Robert J. Boik, 2003. "Principal component models for correlation matrices," Biometrika, Biometrika Trust, vol. 90(3), pages 679-701, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Gabriele Perrone & Gabriele Soffritti, 2024. "Parsimonious Seemingly Unrelated Contaminated Normal Cluster-Weighted Models," Journal of Classification, Springer;The Classification Society, vol. 41(3), pages 533-567, November.
    2. Andrea Cappozzo & Francesca Greselin & Thomas Brendan Murphy, 2020. "A robust approach to model-based classification based on trimming and constraints," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 14(2), pages 327-354, June.
    3. Kim, Nam-Hwui & Browne, Ryan P., 2021. "In the pursuit of sparseness: A new rank-preserving penalty for a finite mixture of factor analyzers," Computational Statistics & Data Analysis, Elsevier, vol. 160(C).
    4. Salvatore D. Tomarchio & Luca Bagnato & Antonio Punzo, 2022. "Model-based clustering via new parsimonious mixtures of heavy-tailed distributions," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 106(2), pages 315-347, June.
    5. Cristina Tortora & Paul D. McNicholas & Ryan P. Browne, 2016. "A mixture of generalized hyperbolic factor analyzers," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 10(4), pages 423-440, December.
    6. Bingkai Wang & Xi Luo & Yi Zhao & Brian Caffo, 2021. "Semiparametric partial common principal component analysis for covariance matrices," Biometrics, The International Biometric Society, vol. 77(4), pages 1175-1186, December.
    7. Ryan P. Browne & Luca Bagnato & Antonio Punzo, 2024. "Parsimony and parameter estimation for mixtures of multivariate leptokurtic-normal distributions," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 18(3), pages 597-625, September.
    8. Diani, Cecilia & Galimberti, Giuliano & Soffritti, Gabriele, 2022. "Multivariate cluster-weighted models based on seemingly unrelated linear regression," Computational Statistics & Data Analysis, Elsevier, vol. 171(C).
    9. Sanjeena Subedi & Paul D. McNicholas, 2021. "A Variational Approximations-DIC Rubric for Parameter Estimation and Mixture Model Selection Within a Family Setting," Journal of Classification, Springer;The Classification Society, vol. 38(1), pages 89-108, April.
    10. Yuhong Wei & Paul McNicholas, 2015. "Mixture model averaging for clustering," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 9(2), pages 197-217, June.
    11. Tang, Yang & Browne, Ryan P. & McNicholas, Paul D., 2015. "Model based clustering of high-dimensional binary data," Computational Statistics & Data Analysis, Elsevier, vol. 87(C), pages 84-101.
    12. Jason Hou-Liu & Ryan P. Browne, 2022. "Chimeral Clustering," Journal of Classification, Springer;The Classification Society, vol. 39(1), pages 171-190, March.
    13. Alex Sharp & Glen Chalatov & Ryan P. Browne, 2023. "A dual subspace parsimonious mixture of matrix normal distributions," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 17(3), pages 801-822, September.
    14. Jason Hou-Liu & Ryan P. Browne, 2022. "Factor and hybrid components for model-based clustering," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 16(2), pages 373-398, June.
    15. Zhao, Jianhua & Jin, Libin & Shi, Lei, 2015. "Mixture model selection via hierarchical BIC," Computational Statistics & Data Analysis, Elsevier, vol. 88(C), pages 139-153.
    16. Cristina Tortora & Brian C. Franczak & Ryan P. Browne & Paul D. McNicholas, 2019. "A Mixture of Coalesced Generalized Hyperbolic Distributions," Journal of Classification, Springer;The Classification Society, vol. 36(1), pages 26-57, April.
    17. Wraith, Darren & Forbes, Florence, 2015. "Location and scale mixtures of Gaussians with flexible tail behaviour: Properties, inference and application to multivariate clustering," Computational Statistics & Data Analysis, Elsevier, vol. 90(C), pages 61-73.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Carlo Cavicchia & Maurizio Vichi & Giorgia Zaccaria, 2022. "Gaussian mixture model with an extended ultrametric covariance structure," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 16(2), pages 399-427, June.
    2. Bouveyron, Charles & Brunet-Saumard, Camille, 2014. "Model-based clustering of high-dimensional data: A review," Computational Statistics & Data Analysis, Elsevier, vol. 71(C), pages 52-78.
    3. Cathy Maugis & Gilles Celeux & Marie-Laure Martin-Magniette, 2009. "Variable Selection for Clustering with Gaussian Mixture Models," Biometrics, The International Biometric Society, vol. 65(3), pages 701-709, September.
    4. Boik, Robert J., 2013. "Model-based principal components of correlation matrices," Journal of Multivariate Analysis, Elsevier, vol. 116(C), pages 310-331.
    5. Kim, Nam-Hwui & Browne, Ryan P., 2021. "In the pursuit of sparseness: A new rank-preserving penalty for a finite mixture of factor analyzers," Computational Statistics & Data Analysis, Elsevier, vol. 160(C).
    6. Charles Bouveyron & Julien Jacques, 2011. "Model-based clustering of time series in group-specific functional subspaces," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 5(4), pages 281-300, December.
    7. Pourahmadi, Mohsen & Daniels, Michael J. & Park, Trevor, 2007. "Simultaneous modelling of the Cholesky decomposition of several covariance matrices," Journal of Multivariate Analysis, Elsevier, vol. 98(3), pages 568-587, March.
    8. Lebret, Rémi & Iovleff, Serge & Langrognet, Florent & Biernacki, Christophe & Celeux, Gilles & Govaert, Gérard, 2015. "Rmixmod: The R Package of the Model-Based Unsupervised, Supervised, and Semi-Supervised Classification Mixmod Library," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 67(i06).
    9. Maugis, C. & Celeux, G. & Martin-Magniette, M.-L., 2011. "Variable selection in model-based discriminant analysis," Journal of Multivariate Analysis, Elsevier, vol. 102(10), pages 1374-1387, November.
    10. Scrucca, Luca, 2011. "Model-based SIR for dimension reduction," Computational Statistics & Data Analysis, Elsevier, vol. 55(11), pages 3010-3026, November.
    11. Jeffrey Andrews & Paul McNicholas, 2014. "Variable Selection for Clustering and Classification," Journal of Classification, Springer;The Classification Society, vol. 31(2), pages 136-153, July.
    12. Krijnen, Wim P., 2006. "Convergence of the sequence of parameters generated by alternating least squares algorithms," Computational Statistics & Data Analysis, Elsevier, vol. 51(2), pages 481-489, November.
    13. Galimberti, Giuliano & Soffritti, Gabriele, 2014. "A multivariate linear regression analysis using finite mixtures of t distributions," Computational Statistics & Data Analysis, Elsevier, vol. 71(C), pages 138-150.
    14. Sanjeena Subedi & Paul McNicholas, 2014. "Variational Bayes approximations for clustering via mixtures of normal inverse Gaussian distributions," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 8(2), pages 167-193, June.
    15. Khadidja Henni & Pierre-Yves Louis & Brigitte Vannier & Ahmed Moussa, 2020. "Is-ClusterMPP: clustering algorithm through point processes and influence space towards high-dimensional data," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 14(3), pages 543-570, September.
    16. Charles Bouveyron & Camille Brunet-Saumard, 2014. "Discriminative variable selection for clustering with the sparse Fisher-EM algorithm," Computational Statistics, Springer, vol. 29(3), pages 489-513, June.
    17. Bouveyron, Charles & Brunet, Camille, 2012. "Theoretical and practical considerations on the convergence properties of the Fisher-EM algorithm," Journal of Multivariate Analysis, Elsevier, vol. 109(C), pages 29-41.
    18. Li, Guoqi & Tang, Pei & Meng, Ziyang & Wen, Changyun & Pei, Jing & Shi, Luping, 2018. "Optimization on matrix manifold based on gradient information and its applications in network control," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 508(C), pages 481-500.
    19. Robert Boik, 2008. "Newton Algorithms for Analytic Rotation: an Implicit Function Approach," Psychometrika, Springer;The Psychometric Society, vol. 73(2), pages 231-259, June.
    20. K. Van Deun & P. J. F. Groenen, 2005. "Majorization Algorithms for Inspecting Circles, Ellipses, Squares, Rectangles, and Rhombi," Operations Research, INFORMS, vol. 53(6), pages 957-967, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:advdac:v:8:y:2014:i:2:p:217-226. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.