IDEAS home Printed from https://ideas.repec.org/p/tin/wpaper/20160073.html
   My bibliography  Save this paper

Forecasting Using Random Subspace Methods

Author

Listed:
  • Tom Boot

    (Erasmus University Rotterdam, the Netherlands)

  • Didier Nibbering

    (Erasmus University Rotterdam, the Netherlands)

Abstract

Random subspace methods are a novel approach to obtain accurate forecasts in high-dimensional regression settings. Forecasts are constructed from random subsets of predictors or randomly weighted predictors. We provide a theoretical justification for these strategies by deriving bounds on their asymptotic mean squared forecast error, which are highly informative on the scenarios where the methods work well. Monte Carlo simulations confirm the theoretical findings and show improvements in predictive accuracy relative to widely used benchmarks. The predictive accuracy on monthly macroeconomic FRED-MD data increases substantially, with random subspace methods outperforming all competing methods for at least 66% of the series.

Suggested Citation

  • Tom Boot & Didier Nibbering, 2016. "Forecasting Using Random Subspace Methods," Tinbergen Institute Discussion Papers 16-073/III, Tinbergen Institute, revised 11 Aug 2017.
  • Handle: RePEc:tin:wpaper:20160073
    as

    Download full text from publisher

    File URL: https://papers.tinbergen.nl/16073.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Kapetanios, George & Marcellino, Massimiliano, 2010. "Factor-GMM estimation with large sets of possibly weak instruments," Computational Statistics & Data Analysis, Elsevier, vol. 54(11), pages 2655-2675, November.
    2. Michael W. McCracken & Serena Ng, 2016. "FRED-MD: A Monthly Database for Macroeconomic Research," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(4), pages 574-589, October.
    3. Inoue, Atsushi & Kilian, Lutz, 2008. "How Useful Is Bagging in Forecasting Economic Time Series? A Case Study of U.S. Consumer Price Inflation," Journal of the American Statistical Association, American Statistical Association, vol. 103, pages 511-522, June.
    4. Bai, Jushan & Ng, Serena, 2008. "Forecasting economic time series using targeted predictors," Journal of Econometrics, Elsevier, vol. 146(2), pages 304-317, October.
    5. Stock, James H. & Watson, Mark W., 2006. "Forecasting with Many Predictors," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 10, pages 515-554, Elsevier.
    6. Koop, Gary & Korobilis, Dimitris & Pettenuzzo, Davide, 2019. "Bayesian compressed vector autoregressions," Journal of Econometrics, Elsevier, vol. 210(1), pages 135-154.
    7. Hansen, Bruce E., 2010. "Averaging estimators for autoregressions with a near unit root," Journal of Econometrics, Elsevier, vol. 158(1), pages 142-155, September.
    8. Elliott, Graham & Gargano, Antonio & Timmermann, Allan, 2013. "Complete subset regressions," Journal of Econometrics, Elsevier, vol. 177(2), pages 357-373.
    9. James H. Stock & Mark W. Watson, 2012. "Generalized Shrinkage Methods for Forecasting Using Many Predictors," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 30(4), pages 481-493, June.
    10. Groen, Jan J.J. & Kapetanios, George, 2016. "Revisiting useful approaches to data-rich macroeconomic forecasting," Computational Statistics & Data Analysis, Elsevier, vol. 100(C), pages 221-239.
    11. Claeskens,Gerda & Hjort,Nils Lid, 2008. "Model Selection and Model Averaging," Cambridge Books, Cambridge University Press, number 9780521852258, September.
    12. Schneider, Matthew J. & Gupta, Sachin, 2016. "Forecasting sales of new and existing products using consumer reviews: A random projections approach," International Journal of Forecasting, Elsevier, vol. 32(2), pages 243-256.
    13. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    14. Friedman, Jerome H. & Hastie, Trevor & Tibshirani, Rob, 2010. "Regularization Paths for Generalized Linear Models via Coordinate Descent," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 33(i01).
    15. Ng, Serena, 2013. "Variable Selection in Predictive Regressions," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 752-789, Elsevier.
    16. Rajarshi Guhaniyogi & David B. Dunson, 2015. "Bayesian Compressed Regression," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(512), pages 1500-1514, December.
    17. Hansen, Bruce E., 2008. "Least-squares forecast averaging," Journal of Econometrics, Elsevier, vol. 146(2), pages 342-350, October.
    18. Keisuke Hirano & Jonathan H. Wright, 2017. "Forecasting With Model Uncertainty: Representations and Risk Reduction," Econometrica, Econometric Society, vol. 85, pages 617-643, March.
    19. Jushan Bai & Serena Ng, 2006. "Confidence Intervals for Diffusion Index Forecasts and Inference for Factor-Augmented Regressions," Econometrica, Econometric Society, vol. 74(4), pages 1133-1150, July.
    20. Stock, James H & Watson, Mark W, 2002. "Macroeconomic Forecasting Using Diffusion Indexes," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(2), pages 147-162, April.
    21. Elliott, Graham & Gargano, Antonio & Timmermann, Allan, 2015. "Complete subset regressions with large-dimensional sets of predictors," Journal of Economic Dynamics and Control, Elsevier, vol. 54(C), pages 86-110.
    22. Stock J.H. & Watson M.W., 2002. "Forecasting Using Principal Components From a Large Number of Predictors," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 1167-1179, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Mohitosh Kejriwal & Xuewen Yu, 2019. "Generalized Forecasr Averaging in Autoregressions with a Near Unit Root," Purdue University Economics Working Papers 1318, Purdue University, Department of Economics.
    2. Christis Katsouris, 2023. "High Dimensional Time Series Regression Models: Applications to Statistical Learning Methods," Papers 2308.16192, arXiv.org.
    3. Kozyrev, Boris, 2024. "Forecast combination and interpretability using random subspace," IWH Discussion Papers 21/2024, Halle Institute for Economic Research (IWH).
    4. Cheng-Feng Wu & Shian-Chang Huang & Chei-Chang Chiou & Tsangyao Chang & Yung-Chih Chen, 2022. "The Relationship Between Economic Growth and Electricity Consumption: Bootstrap ARDL Test with a Fourier Function and Machine Learning Approach," Computational Economics, Springer;Society for Computational Economics, vol. 60(4), pages 1197-1220, December.
    5. Dennis Kant & Andreas Pick & Jasper de Winter, 2022. "Nowcasting GDP using machine learning methods," Working Papers 754, DNB.
    6. Bryan T. Kelly & Asaf Manela & Alan Moreira, 2019. "Text Selection," NBER Working Papers 26517, National Bureau of Economic Research, Inc.
    7. Rachidi Kotchoni & Maxime Leroux & Dalibor Stevanovic, 2019. "Macroeconomic forecast accuracy in a data‐rich environment," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 34(7), pages 1050-1072, November.
    8. Chen, Bin & Maung, Kenwin, 2023. "Time-varying forecast combination for high-dimensional data," Journal of Econometrics, Elsevier, vol. 237(2).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Chen, Qitong & Hong, Yongmiao & Li, Haiqi, 2024. "Time-varying forecast combination for factor-augmented regressions with smooth structural changes," Journal of Econometrics, Elsevier, vol. 240(1).
    2. Kim, Hyun Hak & Swanson, Norman R., 2018. "Mining big data using parsimonious factor, machine learning, variable selection and shrinkage methods," International Journal of Forecasting, Elsevier, vol. 34(2), pages 339-354.
    3. Sagaert, Yves R. & Aghezzaf, El-Houssaine & Kourentzes, Nikolaos & Desmet, Bram, 2018. "Tactical sales forecasting using a very large set of macroeconomic indicators," European Journal of Operational Research, Elsevier, vol. 264(2), pages 558-569.
    4. Smeekes, Stephan & Wijler, Etienne, 2018. "Macroeconomic forecasting using penalized regression methods," International Journal of Forecasting, Elsevier, vol. 34(3), pages 408-430.
    5. Cheng, Xu & Hansen, Bruce E., 2015. "Forecasting with factor-augmented regression: A frequentist model averaging approach," Journal of Econometrics, Elsevier, vol. 186(2), pages 280-293.
    6. Catherine Doz & Peter Fuleky, 2019. "Dynamic Factor Models," Working Papers 2019-4, University of Hawaii Economic Research Organization, University of Hawaii at Manoa.
    7. Kim, Hyun Hak & Swanson, Norman R., 2014. "Forecasting financial and macroeconomic variables using data reduction methods: New empirical evidence," Journal of Econometrics, Elsevier, vol. 178(P2), pages 352-367.
    8. Exterkate, Peter & Groenen, Patrick J.F. & Heij, Christiaan & van Dijk, Dick, 2016. "Nonlinear forecasting with many predictors using kernel ridge regression," International Journal of Forecasting, Elsevier, vol. 32(3), pages 736-753.
    9. Kelly, Bryan & Pruitt, Seth, 2015. "The three-pass regression filter: A new approach to forecasting using many predictors," Journal of Econometrics, Elsevier, vol. 186(2), pages 294-316.
    10. Barbarino, Alessandro & Bura, Efstathia, 2024. "Forecasting Near-equivalence of Linear Dimension Reduction Methods in Large Panels of Macro-variables," Econometrics and Statistics, Elsevier, vol. 31(C), pages 1-18.
    11. Varlam Kutateladze, 2021. "The Kernel Trick for Nonlinear Factor Modeling," Papers 2103.01266, arXiv.org.
    12. Kutateladze, Varlam, 2022. "The kernel trick for nonlinear factor modeling," International Journal of Forecasting, Elsevier, vol. 38(1), pages 165-177.
    13. Eric Hillebrand & Huiyu Huang & Tae-Hwy Lee & Canlin Li, 2018. "Using the Entire Yield Curve in Forecasting Output and Inflation," Econometrics, MDPI, vol. 6(3), pages 1-27, August.
    14. Philippe Goulet Coulombe & Maxime Leroux & Dalibor Stevanovic & Stéphane Surprenant, 2022. "How is machine learning useful for macroeconomic forecasting?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(5), pages 920-964, August.
    15. Tommaso Proietti, 2016. "On the Selection of Common Factors for Macroeconomic Forecasting," Advances in Econometrics, in: Dynamic Factor Models, volume 35, pages 593-628, Emerald Group Publishing Limited.
    16. Pierre Guérin & Danilo Leiva-Leon & Massimiliano Marcellino, 2020. "Markov-Switching Three-Pass Regression Filter," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 38(2), pages 285-302, April.
    17. Barbara Rossi, 2019. "Forecasting in the presence of instabilities: How do we know whether models predict well and how to improve them," Economics Working Papers 1711, Department of Economics and Business, Universitat Pompeu Fabra, revised Jul 2021.
    18. Bräuning, Falk & Koopman, Siem Jan, 2014. "Forecasting macroeconomic variables using collapsed dynamic factor analysis," International Journal of Forecasting, Elsevier, vol. 30(3), pages 572-584.
    19. Koop, Gary & Korobilis, Dimitris & Pettenuzzo, Davide, 2019. "Bayesian compressed vector autoregressions," Journal of Econometrics, Elsevier, vol. 210(1), pages 135-154.
    20. Marine Carrasco & Barbara Rossi, 2016. "In-Sample Inference and Forecasting in Misspecified Factor Models," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(3), pages 313-338, July.

    More about this item

    Keywords

    dimension reduction; forecasting; random subspace;
    All these keywords.

    JEL classification:

    • C32 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes; State Space Models
    • C38 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Classification Methdos; Cluster Analysis; Principal Components; Factor Analysis
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • C55 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Large Data Sets: Modeling and Analysis

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:tin:wpaper:20160073. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Tinbergen Office +31 (0)10-4088900 (email available below). General contact details of provider: https://edirc.repec.org/data/tinbenl.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.