IDEAS home Printed from https://ideas.repec.org/p/ris/albaec/2017_009.html
   My bibliography  Save this paper

Testing for State-Dependent Predictive Ability

Author

Listed:
  • Fossati, Sebastian

    (University of Alberta, Department of Economics)

Abstract

This paper proposes a new test for comparing the out-of sample forecasting performance of two competing models for situations in which the predictive content may be state-dependent (for example, expansion and recession states or low and high volatility states). To apply this test the econometrician is not required to observe when the underlying states shift. The test is simple to implement and accommodates several different cases of interest. An out-of-sample forecasting exercise for US output growth using real-time data illustrates the improvement of this test over previous approaches to perform forecast comparison.

Suggested Citation

  • Fossati, Sebastian, 2017. "Testing for State-Dependent Predictive Ability," Working Papers 2017-9, University of Alberta, Department of Economics.
    • Handle: RePEc:ris:albaec:2017_009
    as

    Download full text from publisher

    File URL: https://sites.ualberta.ca/~econwps/2017/wp2017-09.pdf
    File Function: Full text
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Clark, Todd E. & West, Kenneth D., 2006. "Using out-of-sample mean squared prediction errors to test the martingale difference hypothesis," Journal of Econometrics, Elsevier, vol. 135(1-2), pages 155-186.
    2. West, Kenneth D, 1996. "Asymptotic Inference about Predictive Ability," Econometrica, Econometric Society, vol. 64(5), pages 1067-1084, September.
    3. Antonio Gargano & Davide Pettenuzzo & Allan Timmermann, 2019. "Bond Return Predictability: Economic Value and Links to the Macroeconomy," Management Science, INFORMS, vol. 65(2), pages 508-540, February.
    4. Sebastian Fossati, 2015. "Forecasting US recessions with macro factors," Applied Economics, Taylor & Francis Journals, vol. 47(53), pages 5726-5738, November.
    5. Fossati Sebastian, 2016. "Dating US business cycles with macro factors," Studies in Nonlinear Dynamics & Econometrics, De Gruyter, vol. 20(5), pages 529-547, December.
    6. Luis Filipe Martins & Pierre Perron, 2016. "Improved Tests for Forecast Comparisons in the Presence of Instabilities," Journal of Time Series Analysis, Wiley Blackwell, vol. 37(5), pages 650-659, September.
    7. Diebold, Francis X & Rudebusch, Glenn D, 1996. "Measuring Business Cycles: A Modern Perspective," The Review of Economics and Statistics, MIT Press, vol. 78(1), pages 67-77, February.
    8. Tatevik Sekhposyan & Barbara Rossi, 2008. "Has modelsí forecasting performance for US output growth and inflation changed over time, and when?," Working Papers 09-02, Duke University, Department of Economics.
    9. Dangl, Thomas & Halling, Michael, 2012. "Predictive regressions with time-varying coefficients," Journal of Financial Economics, Elsevier, vol. 106(1), pages 157-181.
    10. Hamilton, James D, 1989. "A New Approach to the Economic Analysis of Nonstationary Time Series and the Business Cycle," Econometrica, Econometric Society, vol. 57(2), pages 357-384, March.
    11. Chauvet, Marcelle, 1998. "An Econometric Characterization of Business Cycle Dynamics with Factor Structure and Regime Switching," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 969-996, November.
    12. Rossi, Barbara & Sekhposyan, Tatevik, 2010. "Have economic models' forecasting performance for US output growth and inflation changed over time, and when?," International Journal of Forecasting, Elsevier, vol. 26(4), pages 808-835, October.
    13. Chauvet, Marcelle & Piger, Jeremy, 2008. "A Comparison of the Real-Time Performance of Business Cycle Dating Methods," Journal of Business & Economic Statistics, American Statistical Association, vol. 26, pages 42-49, January.
    14. Jun Ma & Mark Wohar (ed.), 2014. "Recent Advances in Estimating Nonlinear Models," Springer Books, Springer, edition 127, number 978-1-4614-8060-0, March.
    15. Newey, Whitney & West, Kenneth, 2014. "A simple, positive semi-definite, heteroscedasticity and autocorrelation consistent covariance matrix," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 33(1), pages 125-132.
    16. Hamilton, James D., 1990. "Analysis of time series subject to changes in regime," Journal of Econometrics, Elsevier, vol. 45(1-2), pages 39-70.
    17. Gibbs, Christopher G. & Vasnev, Andrey L., 2024. "Conditionally optimal weights and forward-looking approaches to combining forecasts," International Journal of Forecasting, Elsevier, vol. 40(4), pages 1734-1751.
    18. Maximo Camacho & Gabriel Perez‐Quiros & Pilar Poncela, 2015. "Extracting Nonlinear Signals from Several Economic Indicators," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 30(7), pages 1073-1089, November.
    19. Henkel, Sam James & Martin, J. Spencer & Nardari, Federico, 2011. "Time-varying short-horizon predictability," Journal of Financial Economics, Elsevier, vol. 99(3), pages 560-580, March.
    20. David E. Rapach & Jack K. Strauss & Guofu Zhou, 2010. "Out-of-Sample Equity Premium Prediction: Combination Forecasts and Links to the Real Economy," The Review of Financial Studies, Society for Financial Studies, vol. 23(2), pages 821-862, February.
    21. Chang-Jin Kim & Charles R. Nelson, 1999. "State-Space Models with Regime Switching: Classical and Gibbs-Sampling Approaches with Applications," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262112388, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Granziera, Eleonora & Sekhposyan, Tatevik, 2019. "Predicting relative forecasting performance: An empirical investigation," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1636-1657.
    2. Casini, Alessandro & Perron, Pierre, 2022. "Generalized Laplace Inference In Multiple Change-Points Models," Econometric Theory, Cambridge University Press, vol. 38(1), pages 35-65, February.
    3. Alessandro Casini, 2018. "Tests for Forecast Instability and Forecast Failure under a Continuous Record Asymptotic Framework," Papers 1803.10883, arXiv.org, revised Dec 2018.
    4. Boriss Siliverstovs & Daniel Wochner, 2019. "Recessions as Breadwinner for Forecasters State-Dependent Evaluation of Predictive Ability: Evidence from Big Macroeconomic US Data," KOF Working papers 19-463, KOF Swiss Economic Institute, ETH Zurich.
    5. repec:zbw:bofrdp:2018_023 is not listed on IDEAS
    6. Boriss Siliverstovs & Daniel S. Wochner, 2021. "State‐dependent evaluation of predictive ability," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 40(3), pages 547-574, April.
    7. Granziera, Eleonora & Sekhposyan, Tatevik, 2019. "Predicting relative forecasting performance: An empirical investigation," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1636-1657.
    8. Alessandro Casini & Pierre Perron, 2018. "Structural Breaks in Time Series," Papers 1805.03807, arXiv.org.
    9. Daniel Wochner, 2020. "Dynamic Factor Trees and Forests – A Theory-led Machine Learning Framework for Non-Linear and State-Dependent Short-Term U.S. GDP Growth Predictions," KOF Working papers 20-472, KOF Swiss Economic Institute, ETH Zurich.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Daniel Borup & Jonas N. Eriksen & Mads M. Kjær & Martin Thyrsgaard, 2024. "Predicting Bond Return Predictability," Management Science, INFORMS, vol. 70(2), pages 931-951, February.
    2. Granziera, Eleonora & Sekhposyan, Tatevik, 2019. "Predicting relative forecasting performance: An empirical investigation," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1636-1657.
    3. Odendahl, Florens & Rossi, Barbara & Sekhposyan, Tatevik, 2023. "Evaluating forecast performance with state dependence," Journal of Econometrics, Elsevier, vol. 237(2).
    4. Arabinda Basistha, "undated". "Estimates of Quarterly and Monthly Episodes of Global Recessions: Evidence from Markov-switching Dynamic Factor Models," Working Papers 24-07, Department of Economics, West Virginia University.
    5. Guérin, Pierre & Leiva-Leon, Danilo, 2017. "Model averaging in Markov-switching models: Predicting national recessions with regional data," Economics Letters, Elsevier, vol. 157(C), pages 45-49.
    6. Tsiakas, Ilias & Li, Jiahan & Zhang, Haibin, 2020. "Equity premium prediction and the state of the economy," Journal of Empirical Finance, Elsevier, vol. 58(C), pages 75-95.
    7. Michael T. Owyang & Jeremy Piger & Daniel Soques, 2022. "Contagious switching," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(2), pages 415-432, March.
    8. Daniel de Almeida & Ana-Maria Fuertes & Luiz Koodi Hotta, 2025. "Out-of-Sample Predictability of the Equity Risk Premium," Mathematics, MDPI, vol. 13(2), pages 1-23, January.
    9. Joseph P. Byrne & Dimitris Korobilis & Pinho J. Ribeiro, 2018. "On The Sources Of Uncertainty In Exchange Rate Predictability," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 59(1), pages 329-357, February.
    10. Antonin Aviat & Frédérique Bec & Claude Diebolt & Catherine Doz & Denis Ferrand & Laurent Ferrara & Eric Heyer & Valérie Mignon & Pierre-Alain Pionnier, 2021. "Dating business cycles in France: a reference chronology," SciencePo Working papers Main hal-03373425, HAL.
    11. Rossi, Barbara, 2013. "Advances in Forecasting under Instability," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1203-1324, Elsevier.
    12. Olivier Darné & Laurent Ferrara, 2011. "Identification of Slowdowns and Accelerations for the Euro Area Economy," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 73(3), pages 335-364, June.
    13. Catherine Doz & Peter Fuleky, 2019. "Dynamic Factor Models," Working Papers 2019-4, University of Hawaii Economic Research Organization, University of Hawaii at Manoa.
    14. Yousuf, Kashif & Ng, Serena, 2021. "Boosting high dimensional predictive regressions with time varying parameters," Journal of Econometrics, Elsevier, vol. 224(1), pages 60-87.
    15. Monica Billio & Laurent Ferrara & Dominique Guegan & Gian Luigi Mazzi, 2009. "Evaluation of Nonlinear time-series models for real-time business cycle analysis of the Euro," Documents de travail du Centre d'Economie de la Sorbonne 09053, Université Panthéon-Sorbonne (Paris 1), Centre d'Economie de la Sorbonne.
    16. Camacho, Maximo & Perez-Quiros, Gabriel & Poncela, Pilar, 2018. "Markov-switching dynamic factor models in real time," International Journal of Forecasting, Elsevier, vol. 34(4), pages 598-611.
    17. Dichtl, Hubert & Drobetz, Wolfgang & Neuhierl, Andreas & Wendt, Viktoria-Sophie, 2021. "Data snooping in equity premium prediction," International Journal of Forecasting, Elsevier, vol. 37(1), pages 72-94.
    18. Serena Ng & Jonathan H. Wright, 2013. "Facts and Challenges from the Great Recession for Forecasting and Macroeconomic Modeling," Journal of Economic Literature, American Economic Association, vol. 51(4), pages 1120-1154, December.
    19. Li, Yan & Ng, David T. & Swaminathan, Bhaskaran, 2013. "Predicting market returns using aggregate implied cost of capital," Journal of Financial Economics, Elsevier, vol. 110(2), pages 419-436.
    20. Raffaella Giacomini & Barbara Rossi, 2013. "Forecasting in macroeconomics," Chapters, in: Nigar Hashimzade & Michael A. Thornton (ed.), Handbook of Research Methods and Applications in Empirical Macroeconomics, chapter 17, pages 381-408, Edward Elgar Publishing.

    More about this item

    Keywords

    Forecast Evaluation; Testing; Regime Switching; Structural Change;
    All these keywords.

    JEL classification:

    • C22 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ris:albaec:2017_009. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Joseph Marchand (email available below). General contact details of provider: https://edirc.repec.org/data/deualca.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.