IDEAS home Printed from https://ideas.repec.org/a/eee/intfor/v40y2024i1p202-228.html
   My bibliography  Save this article

Equal predictive ability tests based on panel data with applications to OECD and IMF forecasts

Author

Listed:
  • Akgun, Oguzhan
  • Pirotte, Alain
  • Urga, Giovanni
  • Yang, Zhenlin

Abstract

We propose two types of equal predictive ability (EPA) tests with panels to compare the predictions made by two forecasters. The first type, S-statistics, focuses on the overall EPA hypothesis, which states that the EPA holds, on average, over all panel units and over time. The second type, C-statistics, focuses on the clustered EPA hypothesis where the EPA holds jointly for a fixed number of clusters of panel units. The asymptotic properties of the proposed tests are evaluated under weak and strong cross-sectional dependence. An extensive Monte Carlo simulation shows that the proposed tests have very good finite sample properties, even with little information about the cross-sectional dependence in the data. The proposed framework is applied to compare the economic growth forecasts of the OECD and the IMF, and to evaluate the performance of the consumer price inflation forecasts of the IMF.

Suggested Citation

  • Akgun, Oguzhan & Pirotte, Alain & Urga, Giovanni & Yang, Zhenlin, 2024. "Equal predictive ability tests based on panel data with applications to OECD and IMF forecasts," International Journal of Forecasting, Elsevier, vol. 40(1), pages 202-228.
    • Handle: RePEc:eee:intfor:v:40:y:2024:i:1:p:202-228
    DOI: 10.1016/j.ijforecast.2023.02.001
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0169207023000092
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ijforecast.2023.02.001?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Moscone, F. & Tosetti, Elisa, 2015. "Robust estimation under error cross section dependence," Economics Letters, Elsevier, vol. 133(C), pages 100-104.
    2. Jushan Bai & Serena Ng, 2002. "Determining the Number of Factors in Approximate Factor Models," Econometrica, Econometric Society, vol. 70(1), pages 191-221, January.
    3. West, Kenneth D, 1996. "Asymptotic Inference about Predictive Ability," Econometrica, Econometric Society, vol. 64(5), pages 1067-1084, September.
    4. Moscone, Francesco & Tosetti, Elisa, 2012. "HAC estimation in spatial panels," Economics Letters, Elsevier, vol. 117(1), pages 60-65.
    5. M. Hashem Pesaran & Aman Ullah & Takashi Yamagata, 2008. "A bias-adjusted LM test of error cross-section independence," Econometrics Journal, Royal Economic Society, vol. 11(1), pages 105-127, March.
    6. John C. Driscoll & Aart C. Kraay, 1998. "Consistent Covariance Matrix Estimation With Spatially Dependent Panel Data," The Review of Economics and Statistics, MIT Press, vol. 80(4), pages 549-560, November.
    7. Keane, Michael P & Runkle, David E, 1990. "Testing the Rationality of Price Forecasts: New Evidence from Panel Data," American Economic Review, American Economic Association, vol. 80(4), pages 714-735, September.
    8. Clark, Todd E. & West, Kenneth D., 2007. "Approximately normal tests for equal predictive accuracy in nested models," Journal of Econometrics, Elsevier, vol. 138(1), pages 291-311, May.
    9. Chirok Han & Peter C. B. Phillips & Donggyu Sul, 2017. "Lag length selection in panel autoregression," Econometric Reviews, Taylor & Francis Journals, vol. 36(1-3), pages 225-240, March.
    10. M. Hashem Pesaran, 2015. "Testing Weak Cross-Sectional Dependence in Large Panels," Econometric Reviews, Taylor & Francis Journals, vol. 34(6-10), pages 1089-1117, December.
    11. Clark, Todd E. & McCracken, Michael W., 2015. "Nested forecast model comparisons: A new approach to testing equal accuracy," Journal of Econometrics, Elsevier, vol. 186(1), pages 160-177.
    12. Alexander Chudik & M. Hashem Pesaran & Elisa Tosetti, 2011. "Weak and strong cross‐section dependence and estimation of large panels," Econometrics Journal, Royal Economic Society, vol. 14(1), pages 45-90, February.
    13. Pesaran, M. Hashem & Tosetti, Elisa, 2011. "Large panels with common factors and spatial correlation," Journal of Econometrics, Elsevier, vol. 161(2), pages 182-202, April.
    14. Kelejian, Harry H & Prucha, Ingmar R, 1998. "A Generalized Spatial Two-Stage Least Squares Procedure for Estimating a Spatial Autoregressive Model with Autoregressive Disturbances," The Journal of Real Estate Finance and Economics, Springer, vol. 17(1), pages 99-121, July.
    15. David Turner, 2017. "Designing fan charts for GDP growth forecasts to better reflect downturn risks," OECD Economics Department Working Papers 1428, OECD Publishing.
    16. Phillips, Peter C.B. & Magdalinos, Tassos, 2009. "Unit Root And Cointegrating Limit Theory When Initialization Is In The Infinite Past," Econometric Theory, Cambridge University Press, vol. 25(6), pages 1682-1715, December.
    17. Mariano, Roberto S. & Preve, Daniel, 2012. "Statistical tests for multiple forecast comparison," Journal of Econometrics, Elsevier, vol. 169(1), pages 123-130.
    18. Gneiting, Tilmann, 2011. "Making and Evaluating Point Forecasts," Journal of the American Statistical Association, American Statistical Association, vol. 106(494), pages 746-762.
    19. Raffaella Giacomini & Halbert White, 2006. "Tests of Conditional Predictive Ability," Econometrica, Econometric Society, vol. 74(6), pages 1545-1578, November.
    20. Kim, Min Seong & Sun, Yixiao, 2013. "Heteroskedasticity and spatiotemporal dependence robust inference for linear panel models with fixed effects," Journal of Econometrics, Elsevier, vol. 177(1), pages 85-108.
    21. Clark, Todd E. & McCracken, Michael W., 2001. "Tests of equal forecast accuracy and encompassing for nested models," Journal of Econometrics, Elsevier, vol. 105(1), pages 85-110, November.
    22. Kelejian, Harry H. & Prucha, Ingmar R., 2007. "HAC estimation in a spatial framework," Journal of Econometrics, Elsevier, vol. 140(1), pages 131-154, September.
    23. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    24. T. S. Breusch & A. R. Pagan, 1980. "The Lagrange Multiplier Test and its Applications to Model Specification in Econometrics," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 47(1), pages 239-253.
    25. Vuong, Quang H, 1989. "Likelihood Ratio Tests for Model Selection and Non-nested Hypotheses," Econometrica, Econometric Society, vol. 57(2), pages 307-333, March.
    26. Andrews, Donald W K, 1991. "Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimation," Econometrica, Econometric Society, vol. 59(3), pages 817-858, May.
    27. Axel Dreher & Silvia Marchesi & James Vreeland, 2008. "The political economy of IMF forecasts," Public Choice, Springer, vol. 137(1), pages 145-171, October.
    28. Davies, Anthony & Lahiri, Kajal, 1995. "A new framework for analyzing survey forecasts using three-dimensional panel data," Journal of Econometrics, Elsevier, vol. 68(1), pages 205-227, July.
    29. Jushan Bai, 2003. "Inferential Theory for Factor Models of Large Dimensions," Econometrica, Econometric Society, vol. 71(1), pages 135-171, January.
    30. Jushan Bai & Serena Ng, 2006. "Confidence Intervals for Diffusion Index Forecasts and Inference for Factor-Augmented Regressions," Econometrica, Econometric Society, vol. 74(4), pages 1133-1150, July.
    31. Nigel Pain & Christine Lewis & Thai-Thanh Dang & Yosuke Jin & Pete Richardson, 2014. "OECD Forecasts During and After the Financial Crisis: A Post Mortem," OECD Economics Department Working Papers 1107, OECD Publishing.
    32. repec:hal:journl:peer-00796743 is not listed on IDEAS
    33. Stock J.H. & Watson M.W., 2002. "Forecasting Using Principal Components From a Large Number of Predictors," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 1167-1179, December.
    34. Jansson, Michael, 2002. "Consistent Covariance Matrix Estimation For Linear Processes," Econometric Theory, Cambridge University Press, vol. 18(6), pages 1449-1459, December.
    35. Timmermann, Allan & Zhu, Yinchu, 2019. "Comparing Forecasting Performance with Panel Data," CEPR Discussion Papers 13746, C.E.P.R. Discussion Papers.
    36. Christine Lewis & Nigel Pain, 2014. "Lessons from OECD forecasts during and after the financial crisis," OECD Journal: Economic Studies, OECD Publishing, vol. 2014(1), pages 9-39.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ryan Greenaway-McGrevy & Kade Sorensen, 2021. "A spatial model averaging approach to measuring house prices," Journal of Spatial Econometrics, Springer, vol. 2(1), pages 1-32, December.
    2. Christophe BOUCHER & Wassim LE LANN & Stéphane MATTON & Sessi TOKPAVI, 2021. "Backtesting ESG Ratings," LEO Working Papers / DR LEO 2883, Orleans Economics Laboratory / Laboratoire d'Economie d'Orleans (LEO), University of Orleans.
    3. Qu, Ritong & Timmermann, Allan & Zhu, Yinchu, 2024. "Comparing forecasting performance with panel data," International Journal of Forecasting, Elsevier, vol. 40(3), pages 918-941.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mu-Chun Wang, 2009. "Comparing the DSGE model with the factor model: an out-of-sample forecasting experiment," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 28(2), pages 167-182.
    2. Firmin Doko Tchatoka & Qazi Haque, 2023. "On bootstrapping tests of equal forecast accuracy for nested models," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 42(7), pages 1844-1864, November.
    3. Clark, Todd & McCracken, Michael, 2013. "Advances in Forecast Evaluation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1107-1201, Elsevier.
    4. Raffaella Giacomini & Barbara Rossi, 2013. "Forecasting in macroeconomics," Chapters, in: Nigar Hashimzade & Michael A. Thornton (ed.), Handbook of Research Methods and Applications in Empirical Macroeconomics, chapter 17, pages 381-408, Edward Elgar Publishing.
    5. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    6. Chen, Qitong & Hong, Yongmiao & Li, Haiqi, 2024. "Time-varying forecast combination for factor-augmented regressions with smooth structural changes," Journal of Econometrics, Elsevier, vol. 240(1).
    7. Corradi, Valentina & Fosten, Jack & Gutknecht, Daniel, 2024. "Predictive ability tests with possibly overlapping models," Journal of Econometrics, Elsevier, vol. 241(1).
    8. Min Seong Kim, 2021. "Robust Inference for Diffusion-Index Forecasts with Cross-Sectionally Dependent Data," Working papers 2021-04, University of Connecticut, Department of Economics.
    9. Akgun, Oguzhan & Pirotte, Alain & Urga, Giovanni, 2020. "Forecasting using heterogeneous panels with cross-sectional dependence," International Journal of Forecasting, Elsevier, vol. 36(4), pages 1211-1227.
    10. Jack Fosten, 2016. "Forecast evaluation with factor-augmented models," University of East Anglia School of Economics Working Paper Series 2016-05, School of Economics, University of East Anglia, Norwich, UK..
    11. Nathan Goldstein & Ben‐Zion Zilberfarb, 2023. "The closer we get, the better we are?," Economic Inquiry, Western Economic Association International, vol. 61(2), pages 364-376, April.
    12. Timmermann, Allan & Qu, Ritong & Zhu, Yinchu, 2019. "Do Any Economists Have Superior Forecasting Skills?," CEPR Discussion Papers 14112, C.E.P.R. Discussion Papers.
    13. Timmermann, Allan & Zhu, Yinchu, 2019. "Comparing Forecasting Performance with Panel Data," CEPR Discussion Papers 13746, C.E.P.R. Discussion Papers.
    14. Qu, Ritong & Timmermann, Allan & Zhu, Yinchu, 2023. "Comparing forecasting performance in cross-sections," Journal of Econometrics, Elsevier, vol. 237(2).
    15. Norman R. Swanson & Weiqi Xiong, 2018. "Big data analytics in economics: What have we learned so far, and where should we go from here?," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 51(3), pages 695-746, August.
    16. Rossi, Barbara & Sekhposyan, Tatevik, 2011. "Understanding models' forecasting performance," Journal of Econometrics, Elsevier, vol. 164(1), pages 158-172, September.
    17. Rossi, Barbara, 2013. "Advances in Forecasting under Instability," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1203-1324, Elsevier.
    18. Mayer, Walter J. & Liu, Feng & Dang, Xin, 2017. "Improving the power of the Diebold–Mariano–West test for least squares predictions," International Journal of Forecasting, Elsevier, vol. 33(3), pages 618-626.
    19. West, Kenneth D., 2006. "Forecast Evaluation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 3, pages 99-134, Elsevier.
    20. Shintani, Mototsugu, 2005. "Nonlinear Forecasting Analysis Using Diffusion Indexes: An Application to Japan," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 37(3), pages 517-538, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:intfor:v:40:y:2024:i:1:p:202-228. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/ijforecast .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.