IDEAS home Printed from https://ideas.repec.org/a/eee/econom/v229y2022i2p322-349.html
   My bibliography  Save this article

On LASSO for predictive regression

Author

Listed:
  • Lee, Ji Hyung
  • Shi, Zhentao
  • Gao, Zhan

Abstract

Explanatory variables in a predictive regression typically exhibit low signal strength and various degrees of persistence. Variable selection in such a context is of great importance. In this paper, we explore the pitfalls and possibilities of the LASSO methods in this predictive regression framework. In the presence of stationary, local unit root, and cointegrated predictors, we show that the adaptive LASSO cannot asymptotically eliminate all cointegrating variables with zero regression coefficients. This new finding motivates a novel post-selection adaptive LASSO, which we call the twin adaptive LASSO (TAlasso), to restore variable selection consistency. Accommodating the system of heterogeneous regressors, TAlasso achieves the well-known oracle property. In contrast, conventional LASSO fails to attain coefficient estimation consistency and variable screening in all components simultaneously. We apply these LASSO methods to evaluate the short- and long-horizon predictability of S&P 500 excess returns.

Suggested Citation

  • Lee, Ji Hyung & Shi, Zhentao & Gao, Zhan, 2022. "On LASSO for predictive regression," Journal of Econometrics, Elsevier, vol. 229(2), pages 322-349.
  • Handle: RePEc:eee:econom:v:229:y:2022:i:2:p:322-349
    DOI: 10.1016/j.jeconom.2021.02.002
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S030440762100049X
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.jeconom.2021.02.002?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Liao, Zhipeng & Phillips, Peter C. B., 2015. "Automated Estimation Of Vector Error Correction Models," Econometric Theory, Cambridge University Press, vol. 31(3), pages 581-646, June.
    2. Park, Joon Y, 1992. "Canonical Cointegrating Regressions," Econometrica, Econometric Society, vol. 60(1), pages 119-143, January.
    3. Peter C. B. Phillips & Zhentao Shi, 2021. "Boosting: Why You Can Use The Hp Filter," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 62(2), pages 521-570, May.
    4. Ivo Welch & Amit Goyal, 2008. "A Comprehensive Look at The Empirical Performance of Equity Premium Prediction," The Review of Financial Studies, Society for Financial Studies, vol. 21(4), pages 1455-1508, July.
    5. Zou, Hui, 2006. "The Adaptive Lasso and Its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 1418-1429, December.
    6. Domenico Giannone & Michele Lenza & Giorgio E. Primiceri, 2021. "Economic Predictions With Big Data: The Illusion of Sparsity," Econometrica, Econometric Society, vol. 89(5), pages 2409-2437, September.
    7. Medeiros, Marcelo C. & Mendes, Eduardo F., 2016. "ℓ1-regularization of high-dimensional time-series models with non-Gaussian and heteroskedastic errors," Journal of Econometrics, Elsevier, vol. 191(1), pages 255-271.
    8. Peter C.B. Phillips, 1987. "Multiple Regression with Integrated Time Series," Cowles Foundation Discussion Papers 852, Cowles Foundation for Research in Economics, Yale University.
    9. Shihao Gu & Bryan Kelly & Dacheng Xiu, 2020. "Empirical Asset Pricing via Machine Learning," Review of Finance, European Finance Association, vol. 33(5), pages 2223-2273.
    10. Peter C. B. Phillips & Bruce E. Hansen, 1990. "Statistical Inference in Instrumental Variables Regression with I(1) Processes," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 57(1), pages 99-125.
    11. Inoue, Atsushi & Kilian, Lutz, 2008. "How Useful Is Bagging in Forecasting Economic Time Series? A Case Study of U.S. Consumer Price Inflation," Journal of the American Statistical Association, American Statistical Association, vol. 103, pages 511-522, June.
    12. Koo, Bonsoo & Anderson, Heather M. & Seo, Myung Hwan & Yao, Wenying, 2020. "High-dimensional predictive regression in the presence of cointegration," Journal of Econometrics, Elsevier, vol. 219(2), pages 456-477.
    13. Phillips, Peter C.B. & Ploberger, Werner, 1994. "Posterior Odds Testing for a Unit Root with Data-Based Model Selection," Econometric Theory, Cambridge University Press, vol. 10(3-4), pages 774-808, August.
    14. Liangjun Su & Zhentao Shi & Peter C. B. Phillips, 2016. "Identifying Latent Structures in Panel Data," Econometrica, Econometric Society, vol. 84, pages 2215-2264, November.
    15. Alexei Onatski & Chen Wang, 2021. "Spurious Factor Analysis," Econometrica, Econometric Society, vol. 89(2), pages 591-614, March.
    16. Phillips, Peter C.B. & Lee, Ji Hyung, 2013. "Predictive regression under various degrees of persistence and robust long-horizon regression," Journal of Econometrics, Elsevier, vol. 177(2), pages 250-264.
    17. Peter C. B. Phillips, 2015. "Pitfalls and Possibilities in Predictive Regression," Cowles Foundation Discussion Papers 2003, Cowles Foundation for Research in Economics, Yale University.
    18. Joachim Freyberger & Andreas Neuhierl & Michael Weber & Andrew KarolyiEditor, 2020. "Dissecting Characteristics Nonparametrically," Review of Financial Studies, Society for Financial Studies, vol. 33(5), pages 2326-2377.
    19. Ng, Serena, 2013. "Variable Selection in Predictive Regressions," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 752-789, Elsevier.
    20. Alexandre Belloni & Victor Chernozhukov, 2009. "L1-Penalized Quantile Regression in High-Dimensional Sparse Models," Papers 0904.2931, arXiv.org, revised Sep 2019.
    21. A. Belloni & D. Chen & V. Chernozhukov & C. Hansen, 2012. "Sparse Models and Methods for Optimal Instruments With an Application to Eminent Domain," Econometrica, Econometric Society, vol. 80(6), pages 2369-2429, November.
    22. A. Belloni & V. Chernozhukov & K. Kato, 2015. "Uniform post-selection inference for least absolute deviation regression and other Z-estimation problems," Biometrika, Biometrika Trust, vol. 102(1), pages 77-94.
    23. Lee, Ji Hyung, 2016. "Predictive quantile regression with persistent covariates: IVX-QR approach," Journal of Econometrics, Elsevier, vol. 192(1), pages 105-118.
    24. Martin Lettau & Sydney Ludvigson, 2001. "Consumption, Aggregate Wealth, and Expected Stock Returns," Journal of Finance, American Finance Association, vol. 56(3), pages 815-849, June.
    25. Phillips, P C B, 1987. "Time Series Regression with a Unit Root," Econometrica, Econometric Society, vol. 55(2), pages 277-301, March.
    26. Alexandre Belloni & Victor Chernozhukov & Christian Hansen, 2014. "Inference on Treatment Effects after Selection among High-Dimensional Controlsâ€," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 81(2), pages 608-650.
    27. Phillips, Peter C.B. & Lee, Ji Hyung, 2016. "Robust econometric inference with mixed integrated and mildly explosive regressors," Journal of Econometrics, Elsevier, vol. 192(2), pages 433-450.
    28. Kock, Anders Bredahl, 2016. "Consistent And Conservative Model Selection With The Adaptive Lasso In Stationary And Nonstationary Autoregressions," Econometric Theory, Cambridge University Press, vol. 32(1), pages 243-259, February.
    29. Shihao Gu & Bryan Kelly & Dacheng Xiu, 2020. "Empirical Asset Pricing via Machine Learning," The Review of Financial Studies, Society for Financial Studies, vol. 33(5), pages 2223-2273.
    30. Keisuke Hirano & Jonathan H. Wright, 2017. "Forecasting With Model Uncertainty: Representations and Risk Reduction," Econometrica, Econometric Society, vol. 85, pages 617-643, March.
    31. Alexei Onatski & Chen Wang, 2018. "Alternative Asymptotics for Cointegration Tests in Large VARs," Econometrica, Econometric Society, vol. 86(4), pages 1465-1478, July.
    32. Fan J. & Li R., 2001. "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1348-1360, December.
    33. Zhentao Shi, 2016. "Estimation of Sparse Structural Parameters with Many Endogenous Variables," Econometric Reviews, Taylor & Francis Journals, vol. 35(8-10), pages 1582-1608, December.
    34. Smeekes, Stephan & Wijler, Etienne, 2018. "Macroeconomic forecasting using penalized regression methods," International Journal of Forecasting, Elsevier, vol. 34(3), pages 408-430.
    35. Zhang, Rongmao & Robinson, Peter & Yao, Qiwei, 2019. "Identifying cointegration by eigenanalysis," LSE Research Online Documents on Economics 87431, London School of Economics and Political Science, LSE Library.
    36. Alex Chinco & Adam D. Clark‐Joseph & Mao Ye, 2019. "Sparse Signals in the Cross‐Section of Returns," Journal of Finance, American Finance Association, vol. 74(1), pages 449-492, February.
    37. Serena Ng & Pierre Perron, 2001. "LAG Length Selection and the Construction of Unit Root Tests with Good Size and Power," Econometrica, Econometric Society, vol. 69(6), pages 1519-1554, November.
    38. Phillips, P C B, 1991. "Optimal Inference in Cointegrated Systems," Econometrica, Econometric Society, vol. 59(2), pages 283-306, March.
    39. Kock, Anders Bredahl & Callot, Laurent, 2015. "Oracle inequalities for high dimensional vector autoregressions," Journal of Econometrics, Elsevier, vol. 186(2), pages 325-344.
    40. Mehmet Caner & Hao Helen Zhang, 2014. "Adaptive Elastic Net for Generalized Methods of Moments," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 32(1), pages 30-47, January.
    41. Phillips, P C B, 1987. "Time Series Regression with a Unit Root," Econometrica, Econometric Society, vol. 55(2), pages 277-301, March.
    42. Caner, Mehmet, 2009. "Lasso-Type Gmm Estimator," Econometric Theory, Cambridge University Press, vol. 25(1), pages 270-290, February.
    43. Werner Ploberger & Peter C. B. Phillips, 2003. "Empirical Limits for Time Series Econometric Models," Econometrica, Econometric Society, vol. 71(2), pages 627-673, March.
    44. Jianqing Fan & Jinchi Lv, 2008. "Sure independence screening for ultrahigh dimensional feature space," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 70(5), pages 849-911, November.
    45. Alexandros Kostakis & Tassos Magdalinos & Michalis P. Stamatogiannis, 2015. "Robust Econometric Inference for Stock Return Predictability," The Review of Financial Studies, Society for Financial Studies, vol. 28(5), pages 1506-1553.
    46. Hui Zou & Trevor Hastie, 2005. "Addendum: Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(5), pages 768-768, November.
    47. Hui Zou & Trevor Hastie, 2005. "Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(2), pages 301-320, April.
    48. Graham Elliott, 1998. "On the Robustness of Cointegration Methods when Regressors Almost Have Unit Roots," Econometrica, Econometric Society, vol. 66(1), pages 149-158, January.
    49. Rongmao Zhang & Peter Robinson & Qiwei Yao, 2019. "Identifying Cointegration by Eigenanalysis," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 114(526), pages 916-927, April.
    50. Pollard, David, 1991. "Asymptotics for Least Absolute Deviation Regression Estimators," Econometric Theory, Cambridge University Press, vol. 7(2), pages 186-199, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ziwei Mei & Zhentao Shi, 2022. "On LASSO for High Dimensional Predictive Regression," Papers 2212.07052, arXiv.org, revised Jan 2024.
    2. Christis Katsouris, 2023. "High Dimensional Time Series Regression Models: Applications to Statistical Learning Methods," Papers 2308.16192, arXiv.org.
    3. Ricardo P. Masini & Marcelo C. Medeiros & Eduardo F. Mendes, 2023. "Machine learning advances for time series forecasting," Journal of Economic Surveys, Wiley Blackwell, vol. 37(1), pages 76-111, February.
    4. Chaohua Dong & Jiti Gao & Yundong Tu & Bin Peng, 2023. "Robust M-Estimation for Additive Single-Index Cointegrating Time Series Models," Papers 2301.06631, arXiv.org.
    5. Fan, Rui & Lee, Ji Hyung & Shin, Youngki, 2023. "Predictive quantile regression with mixed roots and increasing dimensions: The ALQR approach," Journal of Econometrics, Elsevier, vol. 237(2).
    6. Etienne Wijler, 2022. "A restricted eigenvalue condition for unit-root non-stationary data," Papers 2208.12990, arXiv.org.
    7. Zhou, Weilun & Gao, Jiti & Harris, David & Kew, Hsein, 2024. "Semi-parametric single-index predictive regression models with cointegrated regressors," Journal of Econometrics, Elsevier, vol. 238(1).
    8. Chaohua Dong & Jiti Gao & Bin Peng & Yundong Tu, 2021. "Multiple-index Nonstationary Time Series Models: Robust Estimation Theory and Practice," Monash Econometrics and Business Statistics Working Papers 18/21, Monash University, Department of Econometrics and Business Statistics.
    9. Pitarakis, Jean-Yves, 2019. "Predictive Regressions," UC3M Working papers. Economics 28554, Universidad Carlos III de Madrid. Departamento de Economía.
    10. Julien Hambuckers & Li Sun & Luca Trapin, 2023. "Measuring tail risk at high-frequency: An $L_1$-regularized extreme value regression approach with unit-root predictors," Papers 2301.01362, arXiv.org.
    11. Campeanu Emilia Mioara & Boitan Iustina Alina & Anghel Dan Gabriel, 2023. "Student engagement and academic performance in pandemic-driven online teaching: An exploratory and machine learning approach," Management & Marketing, Sciendo, vol. 18(s1), pages 315-339, December.
    12. Chaohua Dong & Jiti Gao & Bin Peng & Yundong Tu, 2021. "Multiple-index Nonstationary Time Series Models: Robust Estimation Theory and Practice," Papers 2111.02023, arXiv.org.
    13. David Neto, 2023. "Penalized leads-and-lags cointegrating regression: a simulation study and two empirical applications," Empirical Economics, Springer, vol. 65(2), pages 949-971, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ziwei Mei & Zhentao Shi, 2022. "On LASSO for High Dimensional Predictive Regression," Papers 2212.07052, arXiv.org, revised Jan 2024.
    2. Fan, Rui & Lee, Ji Hyung & Shin, Youngki, 2023. "Predictive quantile regression with mixed roots and increasing dimensions: The ALQR approach," Journal of Econometrics, Elsevier, vol. 237(2).
    3. Ricardo P. Masini & Marcelo C. Medeiros & Eduardo F. Mendes, 2023. "Machine learning advances for time series forecasting," Journal of Economic Surveys, Wiley Blackwell, vol. 37(1), pages 76-111, February.
    4. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    5. Zhan Gao & Ji Hyung Lee & Ziwei Mei & Zhentao Shi, 2024. "Econometric Inference for High Dimensional Predictive Regressions," Papers 2409.10030, arXiv.org, revised Nov 2024.
    6. Tu, Yundong & Xie, Xinling, 2023. "Penetrating sporadic return predictability," Journal of Econometrics, Elsevier, vol. 237(1).
    7. Smeekes, Stephan & Wijler, Etienne, 2021. "An automated approach towards sparse single-equation cointegration modelling," Journal of Econometrics, Elsevier, vol. 221(1), pages 247-276.
    8. Christis Katsouris, 2023. "High Dimensional Time Series Regression Models: Applications to Statistical Learning Methods," Papers 2308.16192, arXiv.org.
    9. Borup, Daniel & Christensen, Bent Jesper & Mühlbach, Nicolaj Søndergaard & Nielsen, Mikkel Slot, 2023. "Targeting predictors in random forest regression," International Journal of Forecasting, Elsevier, vol. 39(2), pages 841-868.
    10. Christis Katsouris, 2023. "Predictability Tests Robust against Parameter Instability," Papers 2307.15151, arXiv.org.
    11. Hoang, Daniel & Wiegratz, Kevin, 2022. "Machine learning methods in finance: Recent applications and prospects," Working Paper Series in Economics 158, Karlsruhe Institute of Technology (KIT), Department of Economics and Management.
    12. Liang, Chong & Schienle, Melanie, 2019. "Determination of vector error correction models in high dimensions," Journal of Econometrics, Elsevier, vol. 208(2), pages 418-441.
    13. Christis Katsouris, 2023. "Structural Break Detection in Quantile Predictive Regression Models with Persistent Covariates," Papers 2302.05193, arXiv.org.
    14. Christis Katsouris, 2023. "Bootstrapping Nonstationary Autoregressive Processes with Predictive Regression Models," Papers 2307.14463, arXiv.org.
    15. Andersen, Torben G. & Varneskov, Rasmus T., 2021. "Consistent inference for predictive regressions in persistent economic systems," Journal of Econometrics, Elsevier, vol. 224(1), pages 215-244.
    16. Demetrescu, Matei & Rodrigues, Paulo M.M. & Taylor, A.M. Robert, 2023. "Transformed regression-based long-horizon predictability tests," Journal of Econometrics, Elsevier, vol. 237(2).
    17. Tomohiro Ando & Naoya Sueishi, 2019. "On the Convergence Rate of the SCAD-Penalized Empirical Likelihood Estimator," Econometrics, MDPI, vol. 7(1), pages 1-14, March.
    18. Shi, Zhentao & Huang, Jingyi, 2023. "Forward-selected panel data approach for program evaluation," Journal of Econometrics, Elsevier, vol. 234(2), pages 512-535.
    19. Ando, Tomohiro & Sueishi, Naoya, 2019. "Regularization parameter selection for penalized empirical likelihood estimator," Economics Letters, Elsevier, vol. 178(C), pages 1-4.
    20. Kasparis, Ioannis & Andreou, Elena & Phillips, Peter C.B., 2015. "Nonparametric predictive regression," Journal of Econometrics, Elsevier, vol. 185(2), pages 468-494.

    More about this item

    Keywords

    Cointegration; Nonstationary time series; Machine learning; Shrinkage estimation; Variable selection;
    All these keywords.

    JEL classification:

    • C22 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • C61 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Optimization Techniques; Programming Models; Dynamic Analysis

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:econom:v:229:y:2022:i:2:p:322-349. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jeconom .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.