IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v57y2013i1p95-111.html
   My bibliography  Save this article

Exact methods for variable selection in principal component analysis: Guide functions and pre-selection

Author

Listed:
  • Pacheco, Joaquín
  • Casado, Silvia
  • Porras, Santiago

Abstract

A variable selection problem is analysed for use in Principal Component Analysis (PCA). In this case, the set of original variables is divided into disjoint groups. The problem resides in the selection of variables, but with the restriction that the set of variables that is selected should contain at least one variable from each group. The objective function under consideration is the sum of the first eigenvalues of the correlation matrix of the subset of selected variables. This problem, with no known prior references, has two further difficulties, in addition to that of the variable selection problem: the evaluation of the objective function and the restriction that the subset of selected variables should also contain elements from all of the groups. Two Branch & Bound methods are proposed to obtain exact solutions that incorporate two strategies: the first one is the use of “fast” guide functions as alternatives to the objective function; the second one is the preselection of variables that help to comply with the latter restriction. From the computational tests, it is seen that both strategies are very efficient and achieve significant reductions in calculation times.

Suggested Citation

  • Pacheco, Joaquín & Casado, Silvia & Porras, Santiago, 2013. "Exact methods for variable selection in principal component analysis: Guide functions and pre-selection," Computational Statistics & Data Analysis, Elsevier, vol. 57(1), pages 95-111.
    • Handle: RePEc:eee:csdana:v:57:y:2013:i:1:p:95-111
    DOI: 10.1016/j.csda.2012.06.014
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167947312002551
    Download Restriction: Full text for ScienceDirect subscribers only.
    File URL: https://libkey.io/10.1016/j.csda.2012.06.014?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. W. J. Krzanowski, 1987. "Selection of Variables to Preserve Multivariate Data Structure, Using Principal Components," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 36(1), pages 22-33, March.
    2. Krzanowski, Wojtek J. & Hand, David J., 2009. "A simple method for screening variables before clustering microarray data," Computational Statistics & Data Analysis, Elsevier, vol. 53(7), pages 2747-2753, May.
    3. Michela Nardo & Michaela Saisana & Andrea Saltelli & Stefano Tarantola & Anders Hoffman & Enrico Giovannini, 2005. "Handbook on Constructing Composite Indicators: Methodology and User Guide," OECD Statistics Working Papers 2005/3, OECD Publishing.
    4. Gatu, Cristian & Yanev, Petko I. & Kontoghiorghes, Erricos J., 2007. "A graph approach to generate all possible regression submodels," Computational Statistics & Data Analysis, Elsevier, vol. 52(2), pages 799-815, October.
    5. Pacheco, Joaquín & Casado, Silvia & Núñez, Laura, 2009. "A variable selection method based on Tabu search for logistic regression models," European Journal of Operational Research, Elsevier, vol. 199(2), pages 506-511, December.
    6. Douglas Steinley & Michael Brusco, 2008. "Selection of Variables in Cluster Analysis: An Empirical Comparison of Eight Procedures," Psychometrika, Springer;The Psychometric Society, vol. 73(1), pages 125-144, March.
    7. Brusco, Michael J. & Steinley, Douglas, 2011. "Exact and approximate algorithms for variable selection in linear discriminant analysis," Computational Statistics & Data Analysis, Elsevier, vol. 55(1), pages 123-131, January.
    8. Li, Baibing & Martin, Elaine B. & Morris, A. Julian, 2002. "On principal component analysis in L1," Computational Statistics & Data Analysis, Elsevier, vol. 40(3), pages 471-474, September.
    9. Kristine Hogarty & Jeffrey Kromrey & John Ferron & Constance Hines, 2004. "Selection of variables in exploratory factor analysis: An empirical comparison of a stepwise and traditional approach," Psychometrika, Springer;The Psychometric Society, vol. 69(4), pages 593-611, December.
    10. I. T. Jolliffe, 1973. "Discarding Variables in a Principal Component Analysis. Ii: Real Data," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 22(1), pages 21-31, March.
    11. Michael Brusco & J. Cradit, 2001. "A variable-selection heuristic for K-means clustering," Psychometrika, Springer;The Psychometric Society, vol. 66(2), pages 249-270, June.
    12. Tangian, Andranik, 2007. "Analysis of the third European survey on working conditions with composite indicators," European Journal of Operational Research, Elsevier, vol. 181(1), pages 468-499, August.
    13. Hofmann, Marc & Gatu, Cristian & Kontoghiorghes, Erricos John, 2007. "Efficient algorithms for computing the best subset regression models for large-scale problems," Computational Statistics & Data Analysis, Elsevier, vol. 52(1), pages 16-29, September.
    14. Michael Brusco & Renu Singh & Douglas Steinley, 2009. "Variable Neighborhood Search Heuristics for Selecting a Subset of Variables in Principal Component Analysis," Psychometrika, Springer;The Psychometric Society, vol. 74(4), pages 705-726, December.
    15. A. Pedro Duarte Silva, 2000. "DISCARDING VARIABLES in PRINCIPAL COMPONENT ANALYSIS : ALGORITHMS for ALL-SUBSETS COMPARISONS," Working Papers de Economia (Economics Working Papers) 02, Católica Porto Business School, Universidade Católica Portuguesa.
    16. Ying Chan & Cheuk Kwan & Tan Shek, 2005. "Quality of Life in Hong Kong: the Cuhk Hong Kong Quality of Life Index," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 71(1), pages 259-289, March.
    17. I. T. Jolliffe, 1972. "Discarding Variables in a Principal Component Analysis. I: Artificial Data," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 21(2), pages 160-173, June.
    18. Pacheco, Joaquin & Casado, Silvia & Nunez, Laura & Gomez, Olga, 2006. "Analysis of new variable selection methods for discriminant analysis," Computational Statistics & Data Analysis, Elsevier, vol. 51(3), pages 1463-1478, December.
    19. António Pedro Duarte Silva, 2002. "Discarding Variables in a Principal Component Analysis: Algorithms for All-Subsets Comparisons," Computational Statistics, Springer, vol. 17(2), pages 251-271, July.
    20. Yutaka Kano & Akira Harada, 2000. "Stepwise variable selection in factor analysis," Psychometrika, Springer;The Psychometric Society, vol. 65(1), pages 7-22, March.
    21. Cadima, Jorge & Cerdeira, J. Orestes & Minhoto, Manuel, 2004. "Computational aspects of algorithms for variable selection in the context of principal components," Computational Statistics & Data Analysis, Elsevier, vol. 47(2), pages 225-236, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Bin Meng & Guotai Chi, 2018. "Evaluation Index System Of Green Industry Based On Maximum Information Content," The Singapore Economic Review (SER), World Scientific Publishing Co. Pte. Ltd., vol. 63(02), pages 229-248, March.
    2. Toleva Borislava, 2022. "ANOVA bootstrapped principal components analysis for logistic regression," Croatian Review of Economic, Business and Social Statistics, Sciendo, vol. 8(1), pages 18-31, June.
    3. Brusco, Michael J., 2014. "A comparison of simulated annealing algorithms for variable selection in principal component analysis and discriminant analysis," Computational Statistics & Data Analysis, Elsevier, vol. 77(C), pages 38-53.
    4. Paz, Alexander & Arteaga, Cristian & Cobos, Carlos, 2019. "Specification of mixed logit models assisted by an optimization framework," Journal of choice modelling, Elsevier, vol. 30(C), pages 50-60.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Brusco, Michael J., 2014. "A comparison of simulated annealing algorithms for variable selection in principal component analysis and discriminant analysis," Computational Statistics & Data Analysis, Elsevier, vol. 77(C), pages 38-53.
    2. Brusco, Michael J. & Steinley, Douglas, 2011. "Exact and approximate algorithms for variable selection in linear discriminant analysis," Computational Statistics & Data Analysis, Elsevier, vol. 55(1), pages 123-131, January.
    3. Michael Brusco & Renu Singh & Douglas Steinley, 2009. "Variable Neighborhood Search Heuristics for Selecting a Subset of Variables in Principal Component Analysis," Psychometrika, Springer;The Psychometric Society, vol. 74(4), pages 705-726, December.
    4. Cumming, J.A. & Wooff, D.A., 2007. "Dimension reduction via principal variables," Computational Statistics & Data Analysis, Elsevier, vol. 52(1), pages 550-565, September.
    5. Fouskakis, D., 2012. "Bayesian variable selection in generalized linear models using a combination of stochastic optimization methods," European Journal of Operational Research, Elsevier, vol. 220(2), pages 414-422.
    6. Jolliffe, Ian, 2022. "A 50-year personal journey through time with principal component analysis," Journal of Multivariate Analysis, Elsevier, vol. 188(C).
    7. Martínez-Ventura, Constanza & Mariño-Martínez, Ricardo & Miguélez-Márquez, Javier, 2023. "Redundancy of Centrality Measures in Financial Market Infrastructures," Latin American Journal of Central Banking (previously Monetaria), Elsevier, vol. 4(4).
    8. Tangian, Andranik S., 2017. "Selection of questions for VAAs and the VAA-based elections," Working Paper Series in Economics 100, Karlsruhe Institute of Technology (KIT), Department of Economics and Management.
    9. Bauer, Jan O. & Drabant, Bernhard, 2021. "Principal loading analysis," Journal of Multivariate Analysis, Elsevier, vol. 184(C).
    10. Gao, Jinxin & Hitchcock, David B., 2010. "James-Stein shrinkage to improve k-means cluster analysis," Computational Statistics & Data Analysis, Elsevier, vol. 54(9), pages 2113-2127, September.
    11. António Pedro Duarte Silva, 2002. "Discarding Variables in a Principal Component Analysis: Algorithms for All-Subsets Comparisons," Computational Statistics, Springer, vol. 17(2), pages 251-271, July.
    12. Cadima, Jorge & Cerdeira, J. Orestes & Minhoto, Manuel, 2004. "Computational aspects of algorithms for variable selection in the context of principal components," Computational Statistics & Data Analysis, Elsevier, vol. 47(2), pages 225-236, September.
    13. Hertrich Markus, 2019. "A Novel Housing Price Misalignment Indicator for Germany," German Economic Review, De Gruyter, vol. 20(4), pages 759-794, December.
    14. Colosimo Bianca Maria & Moya Ester Gutierrez & Moroni Giovanni & Petrò Stefano, 2008. "Statistical Sampling Strategies for Geometric Tolerance Inspection by CMM," Stochastics and Quality Control, De Gruyter, vol. 23(1), pages 109-121, January.
    15. Siniksaran, Enis, 2008. "A geometric interpretation of Mallows' Cp statistic and an alternative plot in variable selection," Computational Statistics & Data Analysis, Elsevier, vol. 52(7), pages 3459-3467, March.
    16. Casado Yusta, Silvia & Nœ–ez Letamendía, Laura & Pacheco Bonrostro, Joaqu’n Antonio, 2018. "Predicting Corporate Failure: The GRASP-LOGIT Model || Predicci—n de la quiebra empresarial: el modelo GRASP-LOGIT," Revista de Métodos Cuantitativos para la Economía y la Empresa = Journal of Quantitative Methods for Economics and Business Administration, Universidad Pablo de Olavide, Department of Quantitative Methods for Economics and Business Administration, vol. 26(1), pages 294-314, Diciembre.
    17. Jérome SARACCO & Marie CHAVENT & Vanessa KUENTZ, 2010. "Clustering of categorical variables around latent variables," Cahiers du GREThA (2007-2019) 2010-02, Groupe de Recherche en Economie Théorique et Appliquée (GREThA).
    18. Postiglione, Paolo & Benedetti, Roberto & Lafratta, Giovanni, 2010. "A regression tree algorithm for the identification of convergence clubs," Computational Statistics & Data Analysis, Elsevier, vol. 54(11), pages 2776-2785, November.
    19. Yang, Guijun & Wang, Zhigang & Deng, Wei, 2010. "Unbiased generalized quasi-regression," Computational Statistics & Data Analysis, Elsevier, vol. 54(3), pages 779-789, March.
    20. Psaradakis, Zacharias & Vávra, Marián, 2014. "On testing for nonlinearity in multivariate time series," Economics Letters, Elsevier, vol. 125(1), pages 1-4.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:57:y:2013:i:1:p:95-111. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.