IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2409.03606.html
   My bibliography  Save this paper

Performance of Empirical Risk Minimization For Principal Component Regression

Author

Listed:
  • Christian Brownlees
  • Gu{dh}mundur Stef'an Gu{dh}mundsson
  • Yaping Wang

Abstract

This paper establishes bounds on the predictive performance of empirical risk minimization for principal component regression. Our analysis is nonparametric, in the sense that the relation between the prediction target and the predictors is not specified. In particular, we do not rely on the assumption that the prediction target is generated by a factor model. In our analysis we consider the cases in which the largest eigenvalues of the covariance matrix of the predictors grow linearly in the number of predictors (strong signal regime) or sublinearly (weak signal regime). The main result of this paper shows that empirical risk minimization for principal component regression is consistent for prediction and, under appropriate conditions, it achieves near-optimal performance in both the strong and weak signal regimes.

Suggested Citation

  • Christian Brownlees & Gu{dh}mundur Stef'an Gu{dh}mundsson & Yaping Wang, 2024. "Performance of Empirical Risk Minimization For Principal Component Regression," Papers 2409.03606, arXiv.org, revised Sep 2024.
    • Handle: RePEc:arx:papers:2409.03606
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2409.03606
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Forni, Mario & Hallin, Marc & Lippi, Marco & Reichlin, Lucrezia, 2005. "The Generalized Dynamic Factor Model: One-Sided Estimation and Forecasting," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 830-840, September.
    2. Forni, Mario & Lippi, Marco, 2001. "The Generalized Dynamic Factor Model: Representation Theory," Econometric Theory, Cambridge University Press, vol. 17(6), pages 1113-1141, December.
    3. Jianqing Fan & Yuan Liao & Martina Mincheva, 2013. "Large covariance estimation by thresholding principal orthogonal complements," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 75(4), pages 603-680, September.
    4. Jushan Bai & Serena Ng, 2002. "Determining the Number of Factors in Approximate Factor Models," Econometrica, Econometric Society, vol. 70(1), pages 191-221, January.
    5. Kock, Anders Bredahl & Callot, Laurent, 2015. "Oracle inequalities for high dimensional vector autoregressions," Journal of Econometrics, Elsevier, vol. 186(2), pages 325-344.
    6. Gonçalves, Sílvia & Perron, Benoit, 2020. "Bootstrapping factor models with cross sectional dependence," Journal of Econometrics, Elsevier, vol. 218(2), pages 476-495.
    7. Bai, Jushan & Ng, Serena, 2019. "Rank regularized estimation of approximate factor models," Journal of Econometrics, Elsevier, vol. 212(1), pages 78-96.
    8. Jushan Bai & Kunpeng Li, 2016. "Maximum Likelihood Estimation and Inference for Approximate Factor Models of High Dimension," The Review of Economics and Statistics, MIT Press, vol. 98(2), pages 298-309, May.
    9. Bai, Jushan & Ng, Serena, 2023. "Approximate factor models with weaker loadings," Journal of Econometrics, Elsevier, vol. 235(2), pages 1893-1916.
    10. Mario Forni & Marc Hallin & Marco Lippi & Lucrezia Reichlin, 2000. "The Generalized Dynamic-Factor Model: Identification And Estimation," The Review of Economics and Statistics, MIT Press, vol. 82(4), pages 540-554, November.
    11. Alexei Onatski, 2010. "Determining the Number of Factors from Empirical Distribution of Eigenvalues," The Review of Economics and Statistics, MIT Press, vol. 92(4), pages 1004-1016, November.
    12. Yoshimasa Uematsu & Takashi Yamagata, 2022. "Estimation of Sparsity-Induced Weak Factor Models," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 41(1), pages 213-227, December.
    13. Seung C. Ahn & Alex R. Horenstein, 2013. "Eigenvalue Ratio Test for the Number of Factors," Econometrica, Econometric Society, vol. 81(3), pages 1203-1227, May.
    14. Jiang, Wenxin & Tanner, Martin A., 2010. "Risk Minimization For Time Series Binary Choice With Variable Selection," Econometric Theory, Cambridge University Press, vol. 26(5), pages 1437-1452, October.
    15. Su, Liangjun & Wang, Xia, 2017. "On time-varying factor models: Estimation and testing," Journal of Econometrics, Elsevier, vol. 198(1), pages 84-101.
    16. Onatski, Alexei, 2012. "Asymptotics of the principal components estimator of large factor models with weakly influential factors," Journal of Econometrics, Elsevier, vol. 168(2), pages 244-258.
    17. Amengual, Dante & Watson, Mark W., 2007. "Consistent Estimation of the Number of Dynamic Factors in a Large N and T Panel," Journal of Business & Economic Statistics, American Statistical Association, vol. 25, pages 91-96, January.
    18. Lam, Clifford & Yao, Qiwei, 2012. "Factor modeling for high-dimensional time series: inference for the number of factors," LSE Research Online Documents on Economics 45684, London School of Economics and Political Science, LSE Library.
    19. Stock J.H. & Watson M.W., 2002. "Forecasting Using Principal Components From a Large Number of Predictors," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 1167-1179, December.
    20. Clifford Lam & Qiwei Yao & Neil Bathia, 2011. "Estimation of latent factors for high-dimensional time series," Biometrika, Biometrika Trust, vol. 98(4), pages 901-918.
    21. Lam, Clifford & Yao, Qiwei & Bathia, Neil, 2011. "Estimation of latent factors for high-dimensional time series," LSE Research Online Documents on Economics 31549, London School of Economics and Political Science, LSE Library.
    22. James H. Stock & Mark W. Watson, 2012. "Generalized Shrinkage Methods for Forecasting Using Many Predictors," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 30(4), pages 481-493, June.
    23. Jushan Bai, 2003. "Inferential Theory for Factor Models of Large Dimensions," Econometrica, Econometric Society, vol. 71(1), pages 135-171, January.
    24. Jushan Bai & Serena Ng, 2006. "Confidence Intervals for Diffusion Index Forecasts and Inference for Factor-Augmented Regressions," Econometrica, Econometric Society, vol. 74(4), pages 1133-1150, July.
    25. Yu, Long & He, Yong & Zhang, Xinsheng, 2019. "Robust factor number specification for large-dimensional elliptical factor model," Journal of Multivariate Analysis, Elsevier, vol. 174(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jianqing Fan & Kunpeng Li & Yuan Liao, 2020. "Recent Developments on Factor Models and its Applications in Econometric Learning," Papers 2009.10103, arXiv.org.
    2. Matteo Barigozzi & Marc Hallin, 2023. "Dynamic Factor Models: a Genealogy," Papers 2310.17278, arXiv.org, revised Jan 2024.
    3. Barigozzi, Matteo & Trapani, Lorenzo, 2020. "Sequential testing for structural stability in approximate factor models," Stochastic Processes and their Applications, Elsevier, vol. 130(8), pages 5149-5187.
    4. Catherine Doz & Peter Fuleky, 2019. "Dynamic Factor Models," Working Papers 2019-4, University of Hawaii Economic Research Organization, University of Hawaii at Manoa.
    5. Poncela, Pilar & Ruiz, Esther & Miranda, Karen, 2021. "Factor extraction using Kalman filter and smoothing: This is not just another survey," International Journal of Forecasting, Elsevier, vol. 37(4), pages 1399-1425.
    6. Barigozzi, Matteo & Hallin, Marc & Luciani, Matteo & Zaffaroni, Paolo, 2024. "Inferential theory for generalized dynamic factor models," Journal of Econometrics, Elsevier, vol. 239(2).
    7. Jin, Sainan & Miao, Ke & Su, Liangjun, 2021. "On factor models with random missing: EM estimation, inference, and cross validation," Journal of Econometrics, Elsevier, vol. 222(1), pages 745-777.
    8. Yuefeng Han & Rong Chen & Cun-Hui Zhang, 2020. "Rank Determination in Tensor Factor Model," Papers 2011.07131, arXiv.org, revised May 2022.
    9. Barigozzi, Matteo & Lippi, Marco & Luciani, Matteo, 2021. "Large-dimensional Dynamic Factor Models: Estimation of Impulse–Response Functions with I(1) cointegrated factors," Journal of Econometrics, Elsevier, vol. 221(2), pages 455-482.
    10. Rua, António, 2017. "A wavelet-based multivariate multiscale approach for forecasting," International Journal of Forecasting, Elsevier, vol. 33(3), pages 581-590.
    11. Bai, Jushan & Liao, Yuan, 2016. "Efficient estimation of approximate factor models via penalized maximum likelihood," Journal of Econometrics, Elsevier, vol. 191(1), pages 1-18.
    12. Stock, J.H. & Watson, M.W., 2016. "Dynamic Factor Models, Factor-Augmented Vector Autoregressions, and Structural Vector Autoregressions in Macroeconomics," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 415-525, Elsevier.
    13. Bai, Jushan & Ng, Serena, 2023. "Approximate factor models with weaker loadings," Journal of Econometrics, Elsevier, vol. 235(2), pages 1893-1916.
    14. Yuefeng Han & Dan Yang & Cun-Hui Zhang & Rong Chen, 2021. "CP Factor Model for Dynamic Tensors," Papers 2110.15517, arXiv.org, revised Apr 2024.
    15. Smeekes, Stephan & Wijler, Etienne, 2018. "Macroeconomic forecasting using penalized regression methods," International Journal of Forecasting, Elsevier, vol. 34(3), pages 408-430.
    16. Fan, Jianqing & Ke, Yuan & Liao, Yuan, 2021. "Augmented factor models with applications to validating market risk factors and forecasting bond risk premia," Journal of Econometrics, Elsevier, vol. 222(1), pages 269-294.
    17. Yuefeng Han & Rong Chen & Dan Yang & Cun-Hui Zhang, 2020. "Tensor Factor Model Estimation by Iterative Projection," Papers 2006.02611, arXiv.org, revised Jul 2024.
    18. Jushan Bai & Serena Ng, 2020. "Simpler Proofs for Approximate Factor Models of Large Dimensions," Papers 2008.00254, arXiv.org.
    19. Matteo Barigozzi & Marco Lippi & Matteo Luciani, 2016. "Non-Stationary Dynamic Factor Models for Large Datasets," Finance and Economics Discussion Series 2016-024, Board of Governors of the Federal Reserve System (U.S.).
    20. Forni, Mario & Hallin, Marc & Lippi, Marco & Zaffaroni, Paolo, 2017. "Dynamic factor models with infinite-dimensional factor space: Asymptotic analysis," Journal of Econometrics, Elsevier, vol. 199(1), pages 74-92.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2409.03606. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.