IDEAS home Printed from https://ideas.repec.org/p/fip/fedkrw/rwp04-10.html
   My bibliography  Save this paper

Improving forecast accuracy by combining recursive and rolling forecasts

Author

Abstract

This paper presents analytical, Monte Carlo, and empirical evidence on the effectiveness of combining recursive and rolling forecasts when linear predictive models are subject to structural change. We first provide a characterization of the bias-variance tradeoff faced when choosing between either the recursive and rolling schemes or a scalar convex combination of the two. From that, we derive pointwise optimal, time-varying and data-dependent observation windows and combining weights designed to minimize mean square forecast error. We then proceed to consider other methods of forecast combination, including Bayesian methods that shrink the rolling forecast to the recursive and Bayesian model averaging. Monte Carlo experiments and several empirical examples indicate that although the recursive scheme is often difficult to beat, when gains can be obtained, some form of shrinkage can often provide improvements in forecast accuracy relative to forecasts made using the recursive scheme or the rolling scheme with a fixed window width.

Suggested Citation

  • Todd E. Clark & Michael W. McCracken, 2004. "Improving forecast accuracy by combining recursive and rolling forecasts," Research Working Paper RWP 04-10, Federal Reserve Bank of Kansas City.
  • Handle: RePEc:fip:fedkrw:rwp04-10
    as

    Download full text from publisher

    File URL: https://www.kansascityfed.org/documents/5375/pdf-RWP04-10.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Orphanides, Athanasios & van Norden, Simon, 2005. "The Reliability of Inflation Forecasts Based on Output Gap Estimates in Real Time," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 37(3), pages 583-601, June.
    2. Gary Koop & Simon M. Potter, 2003. "Forecasting in large macroeconomic panels using Bayesian Model Averaging," Staff Reports 163, Federal Reserve Bank of New York.
    3. Terui, Nobuhiko & van Dijk, Herman K., 2002. "Combined forecasts from linear and nonlinear time series models," International Journal of Forecasting, Elsevier, vol. 18(3), pages 421-438.
    4. Stock, James H. & Watson, Mark W., 1999. "Forecasting inflation," Journal of Monetary Economics, Elsevier, vol. 44(2), pages 293-335, October.
    5. Pesaran, M. Hashem & Timmermann, Allan, 2002. "Market timing and return prediction under model instability," Journal of Empirical Finance, Elsevier, vol. 9(5), pages 495-510, December.
    6. William Poole & Robert H. Rasche, 2002. "Flation," Review, Federal Reserve Bank of St. Louis, vol. 84(Nov), pages 1-6.
      • William Poole, 2002. "Flation," Speech 49, Federal Reserve Bank of St. Louis.
    7. Clarida, Richard H. & Sarno, Lucio & Taylor, Mark P. & Valente, Giorgio, 2003. "The out-of-sample success of term structure models as exchange rate predictors: a step beyond," Journal of International Economics, Elsevier, vol. 60(1), pages 61-83, May.
    8. Hamilton, James D & Kim, Dong Heon, 2002. "A Reexamination of the Predictability of Economic Activity Using the Yield Spread," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 34(2), pages 340-360, May.
    9. John M. Maheu & Stephen Gordon, 2008. "Learning, forecasting and structural breaks," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 23(5), pages 553-583.
    10. N. Gregory Mankiw & Jeffrey A. Miron, 1986. "The Changing Behavior of the Term Structure of Interest Rates," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 101(2), pages 211-228.
    11. Elliott, Graham & Timmermann, Allan, 2002. "Optimal Forecast Combination Under General Loss Functions and Forecast Error Distributions," University of California at San Diego, Economics Working Paper Series qt15r9t2q2, Department of Economics, UC San Diego.
    12. Edgerton, David & Wells, Curt, 1994. "Critical Values for the Cusumsq Statistic in Medium and Large Sized Samples," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 56(3), pages 355-365, August.
    13. Jushan Bai, 1997. "Estimation Of A Change Point In Multiple Regression Models," The Review of Economics and Statistics, MIT Press, vol. 79(4), pages 551-563, November.
    14. Min, Chung-ki & Zellner, Arnold, 1993. "Bayesian and non-Bayesian methods for combining models and forecasts with applications to forecasting international growth rates," Journal of Econometrics, Elsevier, vol. 56(1-2), pages 89-118, March.
    15. Yao, Yi-Ching, 1988. "Estimating the number of change-points via Schwarz' criterion," Statistics & Probability Letters, Elsevier, vol. 6(3), pages 181-189, February.
    16. Stock, James H & Watson, Mark W, 1996. "Evidence on Structural Instability in Macroeconomic Time Series Relations," Journal of Business & Economic Statistics, American Statistical Association, vol. 14(1), pages 11-30, January.
    17. Lange, Joe & Sack, Brian & Whitesell, William, 2003. "Anticipations of Monetary Policy in Financial Markets," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 35(6), pages 889-909, December.
    18. Estrella, Arturo & Hardouvelis, Gikas A, 1991. "The Term Structure as a Predictor of Real Economic Activity," Journal of Finance, American Finance Association, vol. 46(2), pages 555-576, June.
    19. Elliott, Graham & Timmermann, Allan, 2004. "Optimal forecast combinations under general loss functions and forecast error distributions," Journal of Econometrics, Elsevier, vol. 122(1), pages 47-79, September.
    20. Inoue, Atsushi & Rossi, Barbara, 2005. "Recursive Predictability Tests for Real-Time Data," Journal of Business & Economic Statistics, American Statistical Association, vol. 23, pages 336-345, July.
    21. Jushan Bai & Pierre Perron, 2003. "Computation and analysis of multiple structural change models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 18(1), pages 1-22.
    22. Engle, Robert F & Smith, Aaron, 1998. "Stochastic Permanent Breaks," University of California at San Diego, Economics Working Paper Series qt99v0s0zx, Department of Economics, UC San Diego.
    23. Richard H. Clarida & Mark P. Taylor, 1997. "The Term Structure Of Forward Exchange Premiums And The Forecastability Of Spot Exchange Rates: Correcting The Errors," The Review of Economics and Statistics, MIT Press, vol. 79(3), pages 353-361, August.
    24. Raffaella Giacomini & Halbert White, 2006. "Tests of Conditional Predictive Ability," Econometrica, Econometric Society, vol. 74(6), pages 1545-1578, November.
    25. Paye, Bradley S. & Timmermann, Allan, 2002. "How Stable are Financial Prediction Models? Evidence from US and International Stock Market Data," University of California at San Diego, Economics Working Paper Series qt74v515fr, Department of Economics, UC San Diego.
    26. In-Koo Cho & Noah Williams & Thomas J. Sargent, 2002. "Escaping Nash Inflation," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 69(1), pages 1-40.
    27. Robert F. Engle & Aaron D. Smith, 1999. "Stochastic Permanent Breaks," The Review of Economics and Statistics, MIT Press, vol. 81(4), pages 553-574, November.
    28. Andrews, Donald W K, 1993. "Tests for Parameter Instability and Structural Change with Unknown Change Point," Econometrica, Econometric Society, vol. 61(4), pages 821-856, July.
    29. James H. Stock & Mark W.Watson, 2003. "Forecasting Output and Inflation: The Role of Asset Prices," Journal of Economic Literature, American Economic Association, vol. 41(3), pages 788-829, September.
    30. Mark W. Watson & James H. Stock, 2004. "Combination forecasts of output growth in a seven-country data set," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 23(6), pages 405-430.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Rossi, Barbara, 2013. "Advances in Forecasting under Instability," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1203-1324, Elsevier.
    2. Todd E. Clark & Michael W. McCracken, 2006. "Forecasting of small macroeconomic VARs in the presence of instabilities," Research Working Paper RWP 06-09, Federal Reserve Bank of Kansas City.
    3. Todd E. Clark & Michael W. McCracken, 2009. "Combining Forecasts from Nested Models," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 71(3), pages 303-329, June.
    4. Clark, Todd E. & McCracken, Michael W., 2005. "The power of tests of predictive ability in the presence of structural breaks," Journal of Econometrics, Elsevier, vol. 124(1), pages 1-31, January.
    5. John Galbraith & Greg Tkacz, 2007. "How Far Can Forecasting Models Forecast? Forecast Content Horizons for Some Important Macroeconomic Variables," Staff Working Papers 07-1, Bank of Canada.
    6. Geweke, John & Jiang, Yu, 2011. "Inference and prediction in a multiple-structural-break model," Journal of Econometrics, Elsevier, vol. 163(2), pages 172-185, August.
    7. Paye, Bradley S. & Timmermann, Allan, 2006. "Instability of return prediction models," Journal of Empirical Finance, Elsevier, vol. 13(3), pages 274-315, June.
    8. Clark, Todd E. & McCracken, Michael W., 2006. "The Predictive Content of the Output Gap for Inflation: Resolving In-Sample and Out-of-Sample Evidence," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 38(5), pages 1127-1148, August.
    9. Todd E. Clark & Michael W. McCracken, 2002. "Forecast-based model selection in the presence of structural breaks," Research Working Paper RWP 02-05, Federal Reserve Bank of Kansas City.
    10. Benati, Luca & Goodhart, Charles, 2008. "Investigating time-variation in the marginal predictive power of the yield spread," Journal of Economic Dynamics and Control, Elsevier, vol. 32(4), pages 1236-1272, April.
    11. Pesaran, M. Hashem & Pick, Andreas & Pranovich, Mikhail, 2013. "Optimal forecasts in the presence of structural breaks," Journal of Econometrics, Elsevier, vol. 177(2), pages 134-152.
    12. Koo, Bonsoo & Seo, Myung Hwan, 2015. "Structural-break models under mis-specification: Implications for forecasting," Journal of Econometrics, Elsevier, vol. 188(1), pages 166-181.
    13. Hännikäinen Jari, 2017. "Selection of an Estimation Window in the Presence of Data Revisions and Recent Structural Breaks," Journal of Econometric Methods, De Gruyter, vol. 6(1), pages 1-22, January.
    14. Akhter Faroque & William Veloce & Jean-Francois Lamarche, 2012. "Have structural changes eliminated the out-of-sample ability of financial variables to forecast real activity after the mid-1980s? Evidence from the Canadian economy," Applied Economics, Taylor & Francis Journals, vol. 44(30), pages 3965-3985, October.
    15. Clark, Todd & McCracken, Michael, 2013. "Advances in Forecast Evaluation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1107-1201, Elsevier.
    16. Timmermann, Allan, 2006. "Forecast Combinations," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 4, pages 135-196, Elsevier.
    17. M. Hashem Pesaran & Davide Pettenuzzo & Allan Timmermann, 2006. "Forecasting Time Series Subject to Multiple Structural Breaks," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 73(4), pages 1057-1084.
    18. Raffaella Giacomini & Barbara Rossi, 2013. "Forecasting in macroeconomics," Chapters, in: Nigar Hashimzade & Michael A. Thornton (ed.), Handbook of Research Methods and Applications in Empirical Macroeconomics, chapter 17, pages 381-408, Edward Elgar Publishing.
    19. Jiawen Xu & Pierre Perron, 2023. "Forecasting in the presence of in-sample and out-of-sample breaks," Empirical Economics, Springer, vol. 64(6), pages 3001-3035, June.
    20. Raffaella Giacomini & Barbara Rossi, 2006. "How Stable is the Forecasting Performance of the Yield Curve for Output Growth?," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 68(s1), pages 783-795, December.

    More about this item

    Keywords

    Forecasting;

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:fip:fedkrw:rwp04-10. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Zach Kastens (email available below). General contact details of provider: https://edirc.repec.org/data/frbkcus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.