IDEAS home Printed from https://ideas.repec.org/p/ihs/ihsesp/130.html
   My bibliography  Save this paper

Testing for Relative Predictive Accuracy: A Critical Viewpoint

Author

Listed:
  • Kunst, Robert M.

    (Department of Economics and Finance, Institute for Advanced Studies, Vienna)

Abstract

Tests for relative predictive accuracy have become a widespread addendum to forecast comparisons. Many empirical research reports conclude that the difference between the entertained forecasting models is 'insignificant'. This paper collects arguments that cast doubt on the usefulness of relative predictive accuracy tests. The main point is not that test power is too low but that their application is conceptually mistaken. The features are highlighted by means of some Monte Carlo experiments for simple time-series decision problems.

Suggested Citation

  • Kunst, Robert M., 2003. "Testing for Relative Predictive Accuracy: A Critical Viewpoint," Economics Series 130, Institute for Advanced Studies.
    • Handle: RePEc:ihs:ihsesp:130
    as

    Download full text from publisher

    File URL: https://irihs.ihs.ac.at/id/eprint/1489
    File Function: First version, 2003
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Fildes, Robert & Stekler, Herman, 2002. "The state of macroeconomic forecasting," Journal of Macroeconomics, Elsevier, vol. 24(4), pages 435-468, December.
    2. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    3. Tashman, Leonard J., 2000. "Out-of-sample tests of forecasting accuracy: an analysis and review," International Journal of Forecasting, Elsevier, vol. 16(4), pages 437-450.
    4. Robert Kunst, 1997. "Augmented ARCH models for financial time series: stability conditions and empirical evidence," Applied Financial Economics, Taylor & Francis Journals, vol. 7(6), pages 575-586.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Marcelo Moura, 2010. "Testing the Taylor Model Predictability for Exchange Rates in Latin America," Open Economies Review, Springer, vol. 21(4), pages 547-564, September.
    2. Andreas Brunhart, 2014. "Stock Market's Reactions to Revelation of Tax Evasion: An Empirical Assessment," Swiss Journal of Economics and Statistics (SJES), Swiss Society of Economics and Statistics (SSES), vol. 150(III), pages 161-190, September.
    3. Moura, Marcelo L. , & Lima, Adauto R. S. & Mendonça, Rodrigo M., 2008. "Exchange Rate and Fundamentals: The Case of Brazil," Insper Working Papers wpe_114, Insper Working Paper, Insper Instituto de Ensino e Pesquisa.
    4. repec:onb:oenbwp:y::i:89:b:1 is not listed on IDEAS
    5. Jesús Crespo Cuaresma & Ernest Gnan & Doris Ritzberger-Grünwald, 2005. "The term structure as a predictor of real activity and inflation in the euro area: a reassessment," BIS Papers chapters, in: Bank for International Settlements (ed.), Investigating the relationship between the financial and real economy, volume 22, pages 177-92, Bank for International Settlements.
    6. Jean-Stephane Mesonnier, 2011. "The forecasting power of real interest rate gaps: an assessment for the Euro area," Applied Economics, Taylor & Francis Journals, vol. 43(2), pages 153-172.
    7. Maria Antoinette Silgoner, 2005. "An Overview of European Economic Indicators: Great Variety of Data on the Euro Area, Need for More Extensive Coverage of the New EU Member States," Monetary Policy & the Economy, Oesterreichische Nationalbank (Austrian Central Bank), issue 3, pages 66-89.
    8. Jean-Stéphane MESONNIER, 2007. "The predictive content of the real interest rate gap for macroeconomic variables in the euro area," Money Macro and Finance (MMF) Research Group Conference 2006 102, Money Macro and Finance Research Group.
    9. Andreas Breitenfellner & Jesus Crespo Cuaresma, 2008. "Crude Oil Prices and the USD/EUR Exchange Rate," Monetary Policy & the Economy, Oesterreichische Nationalbank (Austrian Central Bank), issue 4.
    10. Mariola Pilatowska, 2011. "Information and Prediction Criteria in Selecting the Forecasting Model," Dynamic Econometric Models, Uniwersytet Mikolaja Kopernika, vol. 11, pages 21-40.
    11. Mésonnier, J-S., 2006. "The Reliability of Macroeconomic Forecasts based on Real Interest Rate Gap Estimates in Real Time: an Assessment for the Euro Area," Working papers 157, Banque de France.
    12. Moser, Gabriel & Rumler, Fabio & Scharler, Johann, 2007. "Forecasting Austrian inflation," Economic Modelling, Elsevier, vol. 24(3), pages 470-480, May.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kelly Burns & Imad Moosa, 2017. "Demystifying the Meese–Rogoff puzzle: structural breaks or measures of forecasting accuracy?," Applied Economics, Taylor & Francis Journals, vol. 49(48), pages 4897-4910, October.
    2. Ericsson, Neil R., 2016. "Eliciting GDP forecasts from the FOMC’s minutes around the financial crisis," International Journal of Forecasting, Elsevier, vol. 32(2), pages 571-583.
    3. Philippe Goulet Coulombe & Maxime Leroux & Dalibor Stevanovic & Stéphane Surprenant, 2022. "How is machine learning useful for macroeconomic forecasting?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(5), pages 920-964, August.
    4. Emilio Zanetti Chini, 2013. "Generalizing smooth transition autoregressions," CREATES Research Papers 2013-32, Department of Economics and Business Economics, Aarhus University.
    5. Artur Tarassow, 2017. "Forecasting growth of U.S. aggregate and household-sector M2 after 2000 using economic uncertainty measures," Macroeconomics and Finance Series 201702, University of Hamburg, Department of Socioeconomics.
    6. Glocker, Christian & Kaniovski, Serguei, 2020. "Structural modeling and forecasting using a cluster of dynamic factor models," MPRA Paper 101874, University Library of Munich, Germany.
    7. Zanetti Chini, Emilio, 2018. "Forecasting dynamically asymmetric fluctuations of the U.S. business cycle," International Journal of Forecasting, Elsevier, vol. 34(4), pages 711-732.
    8. Gianna Boero & Emanuela Marrocu, 2005. "Evaluating non-linear models on point and interval forecasts: an application with exchange rates," BNL Quarterly Review, Banca Nazionale del Lavoro, vol. 58(232), pages 91-120.
    9. Heilemann, Ullrich & Stekler, H. O., 2003. "Has the accuracy of German macroeconomic forecasts improved?," Technical Reports 2003,31, Technische Universität Dortmund, Sonderforschungsbereich 475: Komplexitätsreduktion in multivariaten Datenstrukturen.
    10. Phan, Dinh Hoang Bach & Sharma, Susan Sunila & Narayan, Paresh Kumar, 2015. "Stock return forecasting: Some new evidence," International Review of Financial Analysis, Elsevier, vol. 40(C), pages 38-51.
    11. Alberto Baffigi & Roberto Golinelli & Giuseppe Parigi, 2002. "Real-time GDP forecasting in the euro area," Temi di discussione (Economic working papers) 456, Bank of Italy, Economic Research and International Relations Area.
    12. Wenzel, Lars & Wolf, André, 2013. "Short-term forecasting with business surveys: Evidence for German IHK data at federal state level," HWWI Research Papers 140, Hamburg Institute of International Economics (HWWI).
    13. José R. Maria & Sara Serra, 2008. "Forecasting investment: A fishing contest using survey data," Working Papers w200818, Banco de Portugal, Economics and Research Department.
    14. Imad Moosa & Kelly Burns, 2016. "The random walk as a forecasting benchmark: drift or no drift?," Applied Economics, Taylor & Francis Journals, vol. 48(43), pages 4131-4142, September.
    15. Proietti, Tommaso, 2003. "Forecasting the US unemployment rate," Computational Statistics & Data Analysis, Elsevier, vol. 42(3), pages 451-476, March.
    16. Christopher F. Baum & John Barkoulas, 2006. "Long-memory forecasting of US monetary indices," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 25(4), pages 291-302.
    17. Makridakis, Spyros & Hyndman, Rob J. & Petropoulos, Fotios, 2020. "Forecasting in social settings: The state of the art," International Journal of Forecasting, Elsevier, vol. 36(1), pages 15-28.
    18. Huang, Tao & Fildes, Robert & Soopramanien, Didier, 2019. "Forecasting retailer product sales in the presence of structural change," European Journal of Operational Research, Elsevier, vol. 279(2), pages 459-470.
    19. Lahiri, Kajal & Wang, J. George, 2013. "Evaluating probability forecasts for GDP declines using alternative methodologies," International Journal of Forecasting, Elsevier, vol. 29(1), pages 175-190.
    20. Ashiya, Masahiro, 2007. "Forecast accuracy of the Japanese government: Its year-ahead GDP forecast is too optimistic," Japan and the World Economy, Elsevier, vol. 19(1), pages 68-85, January.

    More about this item

    Keywords

    Information criteria; Forecasting; Hypothesis testing;
    All these keywords.

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C32 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes; State Space Models
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ihs:ihsesp:130. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Doris Szoncsitz (email available below). General contact details of provider: https://edirc.repec.org/data/deihsat.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.