IDEAS home Printed from https://ideas.repec.org/a/wly/japmet/v39y2024i5p850-869.html
   My bibliography  Save this article

Tests for equal forecast accuracy under heteroskedasticity

Author

Listed:
  • David I. Harvey
  • Stephen J. Leybourne
  • Yang Zu

Abstract

Heteroskedasticity is a common feature in empirical time series analysis, and in this paper, we consider the effects of heteroskedasticity on statistical tests for equal forecast accuracy. In such a context, we propose two new Diebold–Mariano‐type tests for equal accuracy that employ nonparametric estimation of the loss differential variance function. We demonstrate that these tests have the potential to achieve power improvements relative to the original Diebold–Mariano test in the presence of heteroskedasticity, for a quite general class of loss differential series. The size validity and potential power superiority of our new tests are studied theoretically and in Monte Carlo simulations. We apply our new tests to competing forecasts of changes in the dollar/sterling exchange rate and find the new tests provide greater evidence of differences in forecast accuracy than the original Diebold–Mariano test, illustrating the value of these new procedures for practitioners.

Suggested Citation

  • David I. Harvey & Stephen J. Leybourne & Yang Zu, 2024. "Tests for equal forecast accuracy under heteroskedasticity," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 39(5), pages 850-869, August.
    • Handle: RePEc:wly:japmet:v:39:y:2024:i:5:p:850-869
    DOI: 10.1002/jae.3050
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/jae.3050
    Download Restriction: no
    File URL: https://libkey.io/10.1002/jae.3050?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. HÄRDLE, Wolfgang & VIEU, Philippe, 1992. "Kernel regression smoothing of time series," LIDAM Reprints CORE 981, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    2. West, Kenneth D, 1996. "Asymptotic Inference about Predictive Ability," Econometrica, Econometric Society, vol. 64(5), pages 1067-1084, September.
    3. Barbara Rossi, 2013. "Exchange Rate Predictability," Journal of Economic Literature, American Economic Association, vol. 51(4), pages 1063-1119, December.
    4. Michael W. McCracken, 2020. "Diverging Tests of Equal Predictive Ability," Econometrica, Econometric Society, vol. 88(4), pages 1753-1754, July.
    5. Graham Elliott & Allan Timmermann, 2016. "Economic Forecasting," Economics Books, Princeton University Press, edition 1, number 10740.
    6. West, Kenneth D & McCracken, Michael W, 1998. "Regression-Based Tests of Predictive Ability," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 817-840, November.
    7. Amisano, Gianni & Giacomini, Raffaella, 2007. "Comparing Density Forecasts via Weighted Likelihood Ratio Tests," Journal of Business & Economic Statistics, American Statistical Association, vol. 25, pages 177-190, April.
    8. Clark, Todd E. & McCracken, Michael W., 2015. "Nested forecast model comparisons: A new approach to testing equal accuracy," Journal of Econometrics, Elsevier, vol. 186(1), pages 160-177.
    9. Bollerslev, Tim, 1986. "Generalized autoregressive conditional heteroskedasticity," Journal of Econometrics, Elsevier, vol. 31(3), pages 307-327, April.
    10. Graham Elliott & Allan Timmermann, 2016. "Forecasting in Economics and Finance," Annual Review of Economics, Annual Reviews, vol. 8(1), pages 81-110, October.
    11. Odendahl, Florens & Rossi, Barbara & Sekhposyan, Tatevik, 2023. "Evaluating forecast performance with state dependence," Journal of Econometrics, Elsevier, vol. 237(2).
    12. repec:bla:jfinan:v:43:y:1988:i:4:p:933-48 is not listed on IDEAS
    13. Giuseppe Cavaliere & Morten Ørregaard Nielsen & A. M. Robert Taylor, 2022. "Adaptive Inference in Heteroscedastic Fractional Time Series Models," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 40(1), pages 50-65, January.
    14. Robert M. De Jong & James Davidson, 2000. "Consistency of Kernel Estimators of Heteroscedastic and Autocorrelated Covariance Matrices," Econometrica, Econometric Society, vol. 68(2), pages 407-424, March.
    15. Meese, R. & Rogoff, K., 1988. "Was It Real? The Exchange Rate-Interest Differential Ralation Over The Modern Floating-Rate Period," Working papers 368, Wisconsin Madison - Social Systems.
    16. Carrasco, Marine & Chen, Xiaohong, 2002. "Mixing And Moment Properties Of Various Garch And Stochastic Volatility Models," Econometric Theory, Cambridge University Press, vol. 18(1), pages 17-39, February.
    17. Xu, Ke-Li & Phillips, Peter C.B., 2008. "Adaptive estimation of autoregressive models with time-varying variances," Journal of Econometrics, Elsevier, vol. 142(1), pages 265-280, January.
    18. Engle, Robert F, 1982. "Autoregressive Conditional Heteroscedasticity with Estimates of the Variance of United Kingdom Inflation," Econometrica, Econometric Society, vol. 50(4), pages 987-1007, July.
    19. Raffaella Giacomini & Halbert White, 2006. "Tests of Conditional Predictive Ability," Econometrica, Econometric Society, vol. 74(6), pages 1545-1578, November.
    20. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    21. Andrews, Donald W K, 1991. "Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimation," Econometrica, Econometric Society, vol. 59(3), pages 817-858, May.
    22. Francis X. Diebold, 2015. "Comparing Predictive Accuracy, Twenty Years Later: A Personal Perspective on the Use and Abuse of Diebold-Mariano Tests," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 33(1), pages 1-1, January.
    23. Meese, Richard A. & Rogoff, Kenneth, 1983. "Empirical exchange rate models of the seventies : Do they fit out of sample?," Journal of International Economics, Elsevier, vol. 14(1-2), pages 3-24, February.
    24. Rossi, Barbara & Sekhposyan, Tatevik, 2010. "Have economic models' forecasting performance for US output growth and inflation changed over time, and when?," International Journal of Forecasting, Elsevier, vol. 26(4), pages 808-835, October.
    25. H. Peter Boswijk & Yang Zu, 2022. "Adaptive Testing for Cointegration With Nonstationary Volatility," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 40(2), pages 744-755, April.
    26. Raffaella Giacomini & Barbara Rossi, 2010. "Forecast comparisons in unstable environments," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(4), pages 595-620.
    27. Wolfgang Härdle & Philippe Vieu, 1992. "Kernel Regression Smoothing Of Time Series," Journal of Time Series Analysis, Wiley Blackwell, vol. 13(3), pages 209-232, May.
    28. Gneiting, Tilmann & Raftery, Adrian E., 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 359-378, March.
    29. Jia Li & Zhipeng Liao & Rogier Quaedvlieg, 2022. "Conditional Superior Predictive Ability," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 89(2), pages 843-875.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Rossi, Barbara, 2013. "Advances in Forecasting under Instability," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1203-1324, Elsevier.
    2. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    3. Barbara Rossi, 2019. "Forecasting in the presence of instabilities: How do we know whether models predict well and how to improve them," Economics Working Papers 1711, Department of Economics and Business, Universitat Pompeu Fabra, revised Jul 2021.
    4. Raffaella Giacomini & Barbara Rossi, 2013. "Forecasting in macroeconomics," Chapters, in: Nigar Hashimzade & Michael A. Thornton (ed.), Handbook of Research Methods and Applications in Empirical Macroeconomics, chapter 17, pages 381-408, Edward Elgar Publishing.
    5. Hambuckers, J. & Ulm, M., 2023. "On the role of interest rate differentials in the dynamic asymmetry of exchange rates," Economic Modelling, Elsevier, vol. 129(C).
    6. Rossi, Barbara & Sekhposyan, Tatevik, 2011. "Understanding models' forecasting performance," Journal of Econometrics, Elsevier, vol. 164(1), pages 158-172, September.
    7. Clark, Todd & McCracken, Michael, 2013. "Advances in Forecast Evaluation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1107-1201, Elsevier.
    8. Jamali, Ibrahim & Yamani, Ehab, 2019. "Out-of-sample exchange rate predictability in emerging markets: Fundamentals versus technical analysis," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 61(C), pages 241-263.
    9. West, Kenneth D., 2006. "Forecast Evaluation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 3, pages 99-134, Elsevier.
    10. Faust, Jon & Wright, Jonathan H., 2013. "Forecasting Inflation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 2-56, Elsevier.
    11. Tae-Hwy Lee & Yong Bao & Burak Saltoğlu, 2007. "Comparing density forecast models Previous versions of this paper have been circulated with the title, 'A Test for Density Forecast Comparison with Applications to Risk Management' since October 2003;," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 26(3), pages 203-225.
    12. Domenico Ferraro & Kenneth S. Rogoff & Barbara Rossi, 2011. "Can oil prices forecast exchange rates?," Working Papers 11-34, Federal Reserve Bank of Philadelphia.
    13. Jie Cheng, 2024. "Evaluating Density Forecasts Using Weighted Multivariate Scores in a Risk Management Context," Computational Economics, Springer;Society for Computational Economics, vol. 64(6), pages 3617-3643, December.
    14. Ferraro, Domenico & Rogoff, Kenneth & Rossi, Barbara, 2015. "Can oil prices forecast exchange rates? An empirical analysis of the relationship between commodity prices and exchange rates," Journal of International Money and Finance, Elsevier, vol. 54(C), pages 116-141.
    15. Clark, Todd E. & West, Kenneth D., 2006. "Using out-of-sample mean squared prediction errors to test the martingale difference hypothesis," Journal of Econometrics, Elsevier, vol. 135(1-2), pages 155-186.
    16. Gaglianone, Wagner Piazza & Marins, Jaqueline Terra Moura, 2017. "Evaluation of exchange rate point and density forecasts: An application to Brazil," International Journal of Forecasting, Elsevier, vol. 33(3), pages 707-728.
    17. Eric Hillebrand & Jakob Mikkelsen & Lars Spreng & Giovanni Urga, 2020. "Exchange Rates and Macroeconomic Fundamentals: Evidence of Instabilities from Time-Varying Factor Loadings," CREATES Research Papers 2020-19, Department of Economics and Business Economics, Aarhus University.
    18. Daniel Borup & Martin Thyrsgaard, 2017. "Statistical tests for equal predictive ability across multiple forecasting methods," CREATES Research Papers 2017-19, Department of Economics and Business Economics, Aarhus University.
    19. William J. Procasky & Anwen Yin, 2022. "Forecasting high‐yield equity and CDS index returns: Does observed cross‐market informational flow have predictive power?," Journal of Futures Markets, John Wiley & Sons, Ltd., vol. 42(8), pages 1466-1490, August.
    20. Calhoun, Gray, 2014. "Out-Of-Sample Comparisons of Overfit Models," Staff General Research Papers Archive 32462, Iowa State University, Department of Economics.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:japmet:v:39:y:2024:i:5:p:850-869. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.interscience.wiley.com/jpages/0883-7252/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.