IDEAS home Printed from https://ideas.repec.org/p/zur/econwp/253.html
   My bibliography  Save this paper

Sharp convergence rates for forward regression in high-dimensional sparse linear models

Author

Listed:
  • Damian Kozbur

Abstract

Forward regression is a statistical model selection and estimation procedure which inductively selects covariates that add predictive power into a working statistical regression model. Once a model is selected, unknown regression parameters are estimated by least squares. This paper analyzes forward regression in high-dimensional sparse linear models. Probabilistic bounds for prediction error norm and number of selected covariates are proved. The analysis in this paper gives sharp rates and does not require β-min or irrepresentability conditions.

Suggested Citation

  • Damian Kozbur, 2017. "Sharp convergence rates for forward regression in high-dimensional sparse linear models," ECON - Working Papers 253, Department of Economics - University of Zurich, revised Apr 2018.
    • Handle: RePEc:zur:econwp:253
    as

    Download full text from publisher

    File URL: https://www.zora.uzh.ch/id/eprint/137364/7/econwp253.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Alexandre Belloni & Victor Chernozhukov & Christian Hansen & Damian Kozbur, 2016. "Inference in High-Dimensional Panel Models With an Application to Gun Control," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(4), pages 590-605, October.
    2. A. Belloni & D. Chen & V. Chernozhukov & C. Hansen, 2012. "Sparse Models and Methods for Optimal Instruments With an Application to Eminent Domain," Econometrica, Econometric Society, vol. 80(6), pages 2369-2429, November.
    3. Bai, Jushan & Ng, Serena, 2008. "Forecasting economic time series using targeted predictors," Journal of Econometrics, Elsevier, vol. 146(2), pages 304-317, October.
    4. Damian Kozbur, 2017. "Testing-Based Forward Model Selection," American Economic Review, American Economic Association, vol. 107(5), pages 266-269, May.
    5. Knight, Keith, 2008. "Shrinkage Estimation For Nearly Singular Designs," Econometric Theory, Cambridge University Press, vol. 24(2), pages 323-337, April.
    6. Jushan Bai & Serena Ng, 2009. "Boosting diffusion indices," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 24(4), pages 607-629.
    7. Wang, Hansheng, 2009. "Forward Regression for Ultra-High Dimensional Variable Screening," Journal of the American Statistical Association, American Statistical Association, vol. 104(488), pages 1512-1524.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Christian Hansen & Damian Kozbur & Sanjog Misra, 2016. "Targeted undersmoothing," ECON - Working Papers 282, Department of Economics - University of Zurich, revised Apr 2018.
    2. Damian Kozbur, 2017. "Testing-Based Forward Model Selection," American Economic Review, American Economic Association, vol. 107(5), pages 266-269, May.
    3. Damian Kozbur, 2020. "Analysis of Testing‐Based Forward Model Selection," Econometrica, Econometric Society, vol. 88(5), pages 2147-2173, September.
    4. Shi, Zhentao & Huang, Jingyi, 2023. "Forward-selected panel data approach for program evaluation," Journal of Econometrics, Elsevier, vol. 234(2), pages 512-535.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Damian Kozbur, 2017. "Testing-Based Forward Model Selection," American Economic Review, American Economic Association, vol. 107(5), pages 266-269, May.
    2. Damian Kozbur, 2013. "Inference in additively separable models with a high-dimensional set of conditioning variables," ECON - Working Papers 284, Department of Economics - University of Zurich, revised Apr 2018.
    3. Shi, Zhentao & Huang, Jingyi, 2023. "Forward-selected panel data approach for program evaluation," Journal of Econometrics, Elsevier, vol. 234(2), pages 512-535.
    4. Zhentao Shi & Jingyi Huang, 2019. "Forward-Selected Panel Data Approach for Program Evaluation," Papers 1908.05894, arXiv.org, revised Apr 2021.
    5. A. Belloni & D. Chen & V. Chernozhukov & C. Hansen, 2012. "Sparse Models and Methods for Optimal Instruments With an Application to Eminent Domain," Econometrica, Econometric Society, vol. 80(6), pages 2369-2429, November.
    6. Hansen, Christian & Liao, Yuan, 2019. "The Factor-Lasso And K-Step Bootstrap Approach For Inference In High-Dimensional Economic Applications," Econometric Theory, Cambridge University Press, vol. 35(3), pages 465-509, June.
    7. Damian Kozbur, 2020. "Analysis of Testing‐Based Forward Model Selection," Econometrica, Econometric Society, vol. 88(5), pages 2147-2173, September.
    8. James M. Carson & Cameron M. Ellis & Robert E. Hoyt & Krzysztof Ostaszewski, 2020. "Sunk Costs and Screening: Two‐Part Tariffs in Life Insurance," Journal of Risk & Insurance, The American Risk and Insurance Association, vol. 87(3), pages 689-718, September.
    9. Peter C. B. Phillips & Zhentao Shi, 2021. "Boosting: Why You Can Use The Hp Filter," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 62(2), pages 521-570, May.
    10. Christian Hansen & Damian Kozbur & Sanjog Misra, 2016. "Targeted undersmoothing," ECON - Working Papers 282, Department of Economics - University of Zurich, revised Apr 2018.
    11. Embaye, Weldensie T. & Zereyesus, Yacob A., 2017. "Measuring the value of housing services in household surveys: an application of machine learning approach," 2017 Annual Meeting, February 4-7, 2017, Mobile, Alabama 252851, Southern Agricultural Economics Association.
    12. Peter C.B. Phillips & Zhentao Shi, 2019. "Boosting the Hodrick-Prescott Filter," Cowles Foundation Discussion Papers 2192, Cowles Foundation for Research in Economics, Yale University.
    13. Chen, Qitong & Hong, Yongmiao & Li, Haiqi, 2024. "Time-varying forecast combination for factor-augmented regressions with smooth structural changes," Journal of Econometrics, Elsevier, vol. 240(1).
    14. Kim, Hyun Hak & Swanson, Norman R., 2018. "Mining big data using parsimonious factor, machine learning, variable selection and shrinkage methods," International Journal of Forecasting, Elsevier, vol. 34(2), pages 339-354.
    15. Norman R. Swanson & Weiqi Xiong, 2018. "Big data analytics in economics: What have we learned so far, and where should we go from here?," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 51(3), pages 695-746, August.
    16. Charles Rahal, 2015. "Housing Market Forecasting with Factor Combinations," Discussion Papers 15-05, Department of Economics, University of Birmingham.
    17. Catherine Doz & Peter Fuleky, 2019. "Dynamic Factor Models," Working Papers halshs-02262202, HAL.
    18. Victor Chernozhukov & Christian Hansen & Martin Spindler, 2015. "Post-Selection and Post-Regularization Inference in Linear Models with Many Controls and Instruments," American Economic Review, American Economic Association, vol. 105(5), pages 486-490, May.
    19. Danquah, Michael & Iddrisu, Abdul Malik & Boakye, Ernest Owusu & Owusu, Solomon, 2021. "Do gender wage differences within households influence women's empowerment and welfare? Evidence from Ghana," Journal of Economic Behavior & Organization, Elsevier, vol. 188(C), pages 916-932.
    20. Holger Breinlich & Valentina Corradi & Nadia Rocha & Michele Ruta & Joao M.C. Santos Silva & Tom Zylkin, 2021. "Machine Learning in International Trade Research ?- Evaluating the Impact of Trade Agreements," School of Economics Discussion Papers 0521, School of Economics, University of Surrey.

    More about this item

    Keywords

    Forward regression; high-dimensional models; sparsity; model selection;
    All these keywords.

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zur:econwp:253. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Severin Oswald (email available below). General contact details of provider: https://edirc.repec.org/data/seizhch.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.