IDEAS home Printed from https://ideas.repec.org/a/oup/biomet/v104y2017i1p97-110..html
   My bibliography  Save this article

Selective factor extraction in high dimensions

Author

Listed:
  • Yiyuan She

Abstract

SUMMARY This paper studies simultaneous feature selection and extraction in supervised and unsupervised learning. We propose and investigate selective reduced rank regression for constructing optimal explanatory factors from a parsimonious subset of input features. The proposed estimators enjoy sharp oracle inequalities, and with a predictive information criterion for model selection, they adapt to unknown sparsity by controlling both rank and row support of the coefficient matrix. A class of algorithms is developed that can accommodate various convex and nonconvex sparsity-inducing penalties, and can be used for rank-constrained variable screening in high-dimensional multivariate data. The paper also showcases applications in macroeconomics and computer vision to demonstrate how low-dimensional data structures can be effectively captured by joint variable selection and projection.

Suggested Citation

  • Yiyuan She, 2017. "Selective factor extraction in high dimensions," Biometrika, Biometrika Trust, vol. 104(1), pages 97-110.
    • Handle: RePEc:oup:biomet:v:104:y:2017:i:1:p:97-110.
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1093/biomet/asw059
    Download Restriction: Access to full text is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Shen, Haipeng & Huang, Jianhua Z., 2008. "Sparse principal component analysis via regularized low rank matrix approximation," Journal of Multivariate Analysis, Elsevier, vol. 99(6), pages 1015-1034, July.
    2. Kun Chen & Kung‐Sik Chan & Nils Chr. Stenseth, 2012. "Reduced rank stochastic regression with a sparse singular value decomposition," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 74(2), pages 203-221, March.
    3. James H. Stock & Mark W. Watson, 2012. "Generalized Shrinkage Methods for Forecasting Using Many Predictors," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 30(4), pages 481-493, June.
    4. She, Yiyuan, 2012. "An iterative algorithm for fitting nonconvex penalized generalized linear models with grouped predictors," Computational Statistics & Data Analysis, Elsevier, vol. 56(10), pages 2976-2990.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yixuan Qiu & Jing Lei & Kathryn Roeder, 2023. "Gradient-based sparse principal component analysis with extensions to online learning," Biometrika, Biometrika Trust, vol. 110(2), pages 339-360.
    2. Aaron J. Molstad & Rohit K. Patra, 2023. "Dimension reduction for integrative survival analysis," Biometrics, The International Biometric Society, vol. 79(3), pages 1610-1623, September.
    3. Wan, Runzhe & Li, Yingying & Lu, Wenbin & Song, Rui, 2024. "Mining the factor zoo: Estimation of latent factor models with sufficient proxies," Journal of Econometrics, Elsevier, vol. 239(2).
    4. Liu, Yong-Jin & Wan, Yuqi & Lin, Lanyu, 2024. "An efficient algorithm for Fantope-constrained sparse principal subspace estimation problem," Applied Mathematics and Computation, Elsevier, vol. 475(C).
    5. Canhong Wen & Zhenduo Li & Ruipeng Dong & Yijin Ni & Wenliang Pan, 2023. "Simultaneous Dimension Reduction and Variable Selection for Multinomial Logistic Regression," INFORMS Journal on Computing, INFORMS, vol. 35(5), pages 1044-1060, September.
    6. Ling Peng & Xiaohui Liu & Xiangyong Tan & Yiweng Zhou & Shihua Luo, 2024. "The statistical rate for support matrix machines under low rankness and row (column) sparsity," Statistical Papers, Springer, vol. 65(7), pages 4567-4598, September.
    7. Mishra, Aditya & Müller, Christian L., 2022. "Robust regression with compositional covariates," Computational Statistics & Data Analysis, Elsevier, vol. 165(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mishra, Aditya & Dey, Dipak K. & Chen, Yong & Chen, Kun, 2021. "Generalized co-sparse factor regression," Computational Statistics & Data Analysis, Elsevier, vol. 157(C).
    2. Hyun Hak Kim, 2013. "Forecasting Macroeconomic Variables Using Data Dimension Reduction Methods: The Case of Korea," Working Papers 2013-26, Economic Research Institute, Bank of Korea.
    3. Smeekes, Stephan & Wijler, Etienne, 2018. "Macroeconomic forecasting using penalized regression methods," International Journal of Forecasting, Elsevier, vol. 34(3), pages 408-430.
    4. Oxana Babecka Kucharcukova & Jan Bruha, 2016. "Nowcasting the Czech Trade Balance," Working Papers 2016/11, Czech National Bank.
    5. Eric Hillebrand & Huiyu Huang & Tae-Hwy Lee & Canlin Li, 2018. "Using the Entire Yield Curve in Forecasting Output and Inflation," Econometrics, MDPI, vol. 6(3), pages 1-27, August.
    6. Kohei Adachi & Nickolay T. Trendafilov, 2016. "Sparse principal component analysis subject to prespecified cardinality of loadings," Computational Statistics, Springer, vol. 31(4), pages 1403-1427, December.
    7. Kim, Hyun Hak & Swanson, Norman R., 2018. "Mining big data using parsimonious factor, machine learning, variable selection and shrinkage methods," International Journal of Forecasting, Elsevier, vol. 34(2), pages 339-354.
    8. Yixuan Qiu & Jing Lei & Kathryn Roeder, 2023. "Gradient-based sparse principal component analysis with extensions to online learning," Biometrika, Biometrika Trust, vol. 110(2), pages 339-360.
    9. Uniejewski, Bartosz & Maciejowska, Katarzyna, 2023. "LASSO principal component averaging: A fully automated approach for point forecast pooling," International Journal of Forecasting, Elsevier, vol. 39(4), pages 1839-1852.
    10. Philippe Goulet Coulombe & Maxime Leroux & Dalibor Stevanovic & Stéphane Surprenant, 2022. "How is machine learning useful for macroeconomic forecasting?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(5), pages 920-964, August.
    11. Norman R. Swanson & Weiqi Xiong, 2018. "Big data analytics in economics: What have we learned so far, and where should we go from here?," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 51(3), pages 695-746, August.
    12. Jushan Bai & Serena Ng, 2020. "Simpler Proofs for Approximate Factor Models of Large Dimensions," Papers 2008.00254, arXiv.org.
    13. Thomas Despois & Catherine Doz, 2022. "Identifying and interpreting the factors in factor models via sparsity : Different approaches," Working Papers halshs-03626503, HAL.
    14. Thomas Despois & Catherine Doz, 2023. "Identifying and interpreting the factors in factor models via sparsity: Different approaches," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 38(4), pages 533-555, June.
    15. Mario Forni & Alessandro Giovannelli & Marco Lippi & Stefano Soccorsi, 2018. "Dynamic factor model with infinite‐dimensional factor space: Forecasting," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 33(5), pages 625-642, August.
    16. Ning Xu & Jian Hong & Timothy C. G. Fisher, 2016. "Finite-sample and asymptotic analysis of generalization ability with an application to penalized regression," Papers 1609.03344, arXiv.org, revised Sep 2016.
    17. Červená, Marianna & Schneider, Martin, 2014. "Short-term forecasting of GDP with a DSGE model augmented by monthly indicators," International Journal of Forecasting, Elsevier, vol. 30(3), pages 498-516.
    18. He Jiang, 2022. "A novel robust structural quadratic forecasting model and applications," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 41(6), pages 1156-1180, September.
    19. Bryan T. Kelly & Asaf Manela & Alan Moreira, 2019. "Text Selection," NBER Working Papers 26517, National Bureau of Economic Research, Inc.
    20. Tommaso Proietti, 2016. "On the Selection of Common Factors for Macroeconomic Forecasting," Advances in Econometrics, in: Dynamic Factor Models, volume 35, pages 593-628, Emerald Group Publishing Limited.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:oup:biomet:v:104:y:2017:i:1:p:97-110.. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Oxford University Press (email available below). General contact details of provider: https://academic.oup.com/biomet .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.