IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2104.12127.html
   My bibliography  Save this paper

Performance of Empirical Risk Minimization for Linear Regression with Dependent Data

Author

Listed:
  • Christian Brownlees
  • Gu{dh}mundur Stef'an Gu{dh}mundsson

Abstract

This paper establishes bounds on the performance of empirical risk minimization for large-dimensional linear regression. We generalize existing results by allowing the data to be dependent and heavy-tailed. The analysis covers both the cases of identically and heterogeneously distributed observations. Our analysis is nonparametric in the sense that the relationship between the regressand and the regressors is not specified. The main results of this paper show that the empirical risk minimizer achieves the optimal performance (up to a logarithmic factor) in a dependent data setting.

Suggested Citation

  • Christian Brownlees & Gu{dh}mundur Stef'an Gu{dh}mundsson, 2021. "Performance of Empirical Risk Minimization for Linear Regression with Dependent Data," Papers 2104.12127, arXiv.org, revised May 2023.
    • Handle: RePEc:arx:papers:2104.12127
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2104.12127
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Liao, Zhipeng & Phillips, Peter C. B., 2015. "Automated Estimation Of Vector Error Correction Models," Econometric Theory, Cambridge University Press, vol. 31(3), pages 581-646, June.
    2. Hansen, Bruce E., 2008. "Uniform Convergence Rates For Kernel Estimation With Dependent Data," Econometric Theory, Cambridge University Press, vol. 24(3), pages 726-748, June.
    3. Meitz, Mika & Saikkonen, Pentti, 2008. "Ergodicity, Mixing, And Existence Of Moments Of A Class Of Markov Models With Applications To Garch And Acd Models," Econometric Theory, Cambridge University Press, vol. 24(5), pages 1291-1320, October.
    4. Medeiros, Marcelo C. & Mendes, Eduardo F., 2016. "ℓ1-regularization of high-dimensional time-series models with non-Gaussian and heteroskedastic errors," Journal of Econometrics, Elsevier, vol. 191(1), pages 255-271.
    5. Forni, Mario & Lippi, Marco, 2001. "The Generalized Dynamic Factor Model: Representation Theory," Econometric Theory, Cambridge University Press, vol. 17(6), pages 1113-1141, December.
    6. Kristensen, Dennis, 2009. "Uniform Convergence Rates Of Kernel Estimators With Heterogeneous Dependent Data," Econometric Theory, Cambridge University Press, vol. 25(5), pages 1433-1445, October.
    7. Jushan Bai & Serena Ng, 2002. "Determining the Number of Factors in Approximate Factor Models," Econometrica, Econometric Society, vol. 70(1), pages 191-221, January.
    8. Andrews, Donald W K, 1991. "Asymptotic Normality of Series Estimators for Nonparametric and Semiparametric Regression Models," Econometrica, Econometric Society, vol. 59(2), pages 307-345, March.
    9. Kock, Anders Bredahl & Callot, Laurent, 2015. "Oracle inequalities for high dimensional vector autoregressions," Journal of Econometrics, Elsevier, vol. 186(2), pages 325-344.
    10. Miao, Ke & Phillips, Peter C.B. & Su, Liangjun, 2023. "High-dimensional VARs with common factors," Journal of Econometrics, Elsevier, vol. 233(1), pages 155-183.
    11. Mario Forni & Marc Hallin & Marco Lippi & Lucrezia Reichlin, 2000. "The Generalized Dynamic-Factor Model: Identification And Estimation," The Review of Economics and Statistics, MIT Press, vol. 82(4), pages 540-554, November.
    12. Jiang, Wenxin & Tanner, Martin A., 2010. "Risk Minimization For Time Series Binary Choice With Variable Selection," Econometric Theory, Cambridge University Press, vol. 26(5), pages 1437-1452, October.
    13. Qi Li & Jeffrey Scott Racine, 2006. "Nonparametric Econometrics: Theory and Practice," Economics Books, Princeton University Press, edition 1, volume 1, number 8355.
    14. Liebscher, Eckhard, 1996. "Strong convergence of sums of [alpha]-mixing random variables with applications to density estimation," Stochastic Processes and their Applications, Elsevier, vol. 65(1), pages 69-80, December.
    15. Qi Li & Jeffrey Scott Racine, 2006. "Density Estimation, from Nonparametric Econometrics: Theory and Practice," Introductory Chapters, in: Nonparametric Econometrics: Theory and Practice, Princeton University Press.
    16. Onatski, Alexei, 2012. "Asymptotics of the principal components estimator of large factor models with weakly influential factors," Journal of Econometrics, Elsevier, vol. 168(2), pages 244-258.
    17. Stock J.H. & Watson M.W., 2002. "Forecasting Using Principal Components From a Large Number of Predictors," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 1167-1179, December.
    18. Newey, Whitney K., 1997. "Convergence rates and asymptotic normality for series estimators," Journal of Econometrics, Elsevier, vol. 79(1), pages 147-168, July.
    19. Andrii Babii & Eric Ghysels & Jonas Striaukas, 2024. "High-Dimensional Granger Causality Tests with an Application to VIX and News," Journal of Financial Econometrics, Oxford University Press, vol. 22(3), pages 605-635.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Toru Kitagawa & Weining Wang & Mengshan Xu, 2022. "Policy Choice in Time Series by Empirical Welfare Maximization," Papers 2205.03970, arXiv.org, revised Jun 2023.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Smeekes, Stephan & Wijler, Etienne, 2018. "Macroeconomic forecasting using penalized regression methods," International Journal of Forecasting, Elsevier, vol. 34(3), pages 408-430.
    2. Christian Brownlees & Gu{dh}mundur Stef'an Gu{dh}mundsson & Yaping Wang, 2024. "Performance of Empirical Risk Minimization For Principal Component Regression," Papers 2409.03606, arXiv.org, revised Sep 2024.
    3. Jonas Krampe & Luca Margaritella, 2021. "Factor Models with Sparse VAR Idiosyncratic Components," Papers 2112.07149, arXiv.org, revised May 2022.
    4. Chen, Xiaohong & Christensen, Timothy M., 2015. "Optimal uniform convergence rates and asymptotic normality for series estimators under weak dependence and weak conditions," Journal of Econometrics, Elsevier, vol. 188(2), pages 447-465.
    5. Barigozzi, Matteo & Trapani, Lorenzo, 2020. "Sequential testing for structural stability in approximate factor models," Stochastic Processes and their Applications, Elsevier, vol. 130(8), pages 5149-5187.
    6. Rua, António, 2017. "A wavelet-based multivariate multiscale approach for forecasting," International Journal of Forecasting, Elsevier, vol. 33(3), pages 581-590.
    7. Bai, Jushan & Liao, Yuan, 2016. "Efficient estimation of approximate factor models via penalized maximum likelihood," Journal of Econometrics, Elsevier, vol. 191(1), pages 1-18.
    8. Bai, Jushan & Ng, Serena, 2023. "Approximate factor models with weaker loadings," Journal of Econometrics, Elsevier, vol. 235(2), pages 1893-1916.
    9. Matteo Barigozzi & Christian Brownlees, 2019. "NETS: Network estimation for time series," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 34(3), pages 347-364, April.
    10. Barigozzi, Matteo & Hallin, Marc & Luciani, Matteo & Zaffaroni, Paolo, 2024. "Inferential theory for generalized dynamic factor models," Journal of Econometrics, Elsevier, vol. 239(2).
    11. Matteo Barigozzi & Marc Hallin, 2017. "A network analysis of the volatility of high dimensional financial series," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 66(3), pages 581-605, April.
    12. Alonso, Andrés M. & Galeano, Pedro & Peña, Daniel, 2020. "A robust procedure to build dynamic factor models with cluster structure," Journal of Econometrics, Elsevier, vol. 216(1), pages 35-52.
    13. Luciani, Matteo, 2014. "Forecasting with approximate dynamic factor models: The role of non-pervasive shocks," International Journal of Forecasting, Elsevier, vol. 30(1), pages 20-29.
    14. Matteo Barigozzi & Marc Hallin, 2015. "Networks, Dynamic Factors, and the Volatility Analysis of High-Dimensional Financial Series," Papers 1510.05118, arXiv.org, revised Jul 2016.
    15. Xiaohong Chen & Timothy M. Christensen, 2014. "Optimal uniform convergence rates and asymptotic normality for series estimators under weak dependence and weak conditions," CeMMAP working papers 46/14, Institute for Fiscal Studies.
    16. Lippi, Marco & Deistler, Manfred & Anderson, Brian, 2023. "High-Dimensional Dynamic Factor Models: A Selective Survey and Lines of Future Research," Econometrics and Statistics, Elsevier, vol. 26(C), pages 3-16.
    17. Fan, Jianqing & Ke, Yuan & Liao, Yuan, 2021. "Augmented factor models with applications to validating market risk factors and forecasting bond risk premia," Journal of Econometrics, Elsevier, vol. 222(1), pages 269-294.
    18. Philipp Gersing & Christoph Rust & Manfred Deistler, 2023. "Weak Factors are Everywhere," Papers 2307.10067, arXiv.org, revised Jan 2024.
    19. Matteo Barigozzi & Marc Hallin & Stefano Soccorsi, 2017. "Identification of Global and National Shocks in International Financial Markets via General Dynamic Factor Models," Working Papers ECARES ECARES 2017-10, ULB -- Universite Libre de Bruxelles.
    20. Cheng, Xu & Hansen, Bruce E., 2015. "Forecasting with factor-augmented regression: A frequentist model averaging approach," Journal of Econometrics, Elsevier, vol. 186(2), pages 280-293.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2104.12127. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.