IDEAS home Printed from https://ideas.repec.org/a/spr/italej/v1y2015i3p333-351.html
   My bibliography  Save this article

How Good are Out of Sample Forecasting Tests on DSGE Models?

Author

Listed:
  • Patrick Minford
  • Yongdeng Xu
  • Peng Zhou

Abstract

Out-of-sample forecasting tests of DSGE models against time-series benchmarks such as an unrestricted VAR are increasingly used to check (a) the specification and (b) the forecasting capacity of these models. We carry out a Monte Carlo experiment on a widely-used DSGE model to investigate the power of these tests. We find that in specification testing they have weak power relative to an in-sample indirect inference test; this implies that a DSGE model may be badly mis-specified and still improve forecasts from an unrestricted VAR. In testing forecasting capacity they also have quite weak power, particularly on the lefthand tail. By contrast a model that passes an indirect inference test of specification will almost definitely also improve on VAR forecasts. Copyright Società Italiana degli Economisti (Italian Economic Association) 2015

Suggested Citation

  • Patrick Minford & Yongdeng Xu & Peng Zhou, 2015. "How Good are Out of Sample Forecasting Tests on DSGE Models?," Italian Economic Journal: A Continuation of Rivista Italiana degli Economisti and Giornale degli Economisti, Springer;Società Italiana degli Economisti (Italian Economic Association), vol. 1(3), pages 333-351, November.
    • Handle: RePEc:spr:italej:v:1:y:2015:i:3:p:333-351
    DOI: 10.1007/s40797-015-0020-9
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s40797-015-0020-9
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s40797-015-0020-9?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Clark, Todd E. & West, Kenneth D., 2006. "Using out-of-sample mean squared prediction errors to test the martingale difference hypothesis," Journal of Econometrics, Elsevier, vol. 135(1-2), pages 155-186.
    2. West, Kenneth D, 1996. "Asymptotic Inference about Predictive Ability," Econometrica, Econometric Society, vol. 64(5), pages 1067-1084, September.
    3. Michael Wickens, 2014. "How Useful are DSGE Macroeconomic Models for Forecasting?," Open Economies Review, Springer, vol. 25(1), pages 171-193, February.
    4. Michael P. Clements & David F. Hendry, 2005. "Evaluating a Model by Forecast Performance," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 67(s1), pages 931-956, December.
    5. Minford, Patrick & Meenagh, David & Le, Vo Phuong Mai, 2012. "What causes banking crises? An empirical investigation," CEPR Discussion Papers 9057, C.E.P.R. Discussion Papers.
    6. Le, Vo Phuong Mai & Meenagh, David & Minford, Patrick & Wickens, Michael, 2011. "How much nominal rigidity is there in the US economy? Testing a new Keynesian DSGE model using indirect inference," Journal of Economic Dynamics and Control, Elsevier, vol. 35(12), pages 2078-2104.
    7. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    8. Rossi, Barbara & Gürkaynak, Refet & Kısacıkoğlu, Burçin, 2013. "Do DSGE Models Forecast More Accurately Out-of-Sample than VAR Models?," CEPR Discussion Papers 9576, C.E.P.R. Discussion Papers.
    9. Clark, Todd E. & West, Kenneth D., 2007. "Approximately normal tests for equal predictive accuracy in nested models," Journal of Econometrics, Elsevier, vol. 138(1), pages 291-311, May.
    10. Rochelle M. Edge & Michael T. Kiley & Jean-Philippe Laforte, 2010. "A comparison of forecast performance between Federal Reserve staff forecasts, simple reduced-form models, and a DSGE model," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(4), pages 720-754.
    11. Michel Juillard, 2001. "DYNARE: A program for the simulation of rational expectation models," Computing in Economics and Finance 2001 213, Society for Computational Economics.
    12. Kenneth S. Rogoff & Vania Stavrakeva, 2008. "The Continuing Puzzle of Short Horizon Exchange Rate Forecasting," NBER Working Papers 14071, National Bureau of Economic Research, Inc.
    13. Frank Smets & Rafael Wouters, 2007. "Shocks and Frictions in US Business Cycles: A Bayesian DSGE Approach," American Economic Review, American Economic Association, vol. 97(3), pages 586-606, June.
    14. Ince, Onur, 2014. "Forecasting exchange rates out-of-sample with panel methods and real-time data," Journal of International Money and Finance, Elsevier, vol. 43(C), pages 1-18.
    15. Le, Vo Phuong Mai & Meenagh, David & Minford, Patrick & Wickens, Michael, 2012. "Testing DSGE models by Indirect inference and other methods: some Monte Carlo experiments," Cardiff Economics Working Papers E2012/15, Cardiff University, Cardiff Business School, Economics Section.
    16. Warne, Anders & Coenen, Günter & Christoffel, Kai, 2010. "Forecasting with DSGE models," Working Paper Series 1185, European Central Bank.
    17. Raffaella Giacomini & Barbara Rossi, 2010. "Forecast comparisons in unstable environments," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(4), pages 595-620.
    18. Rochelle M. Edge & Refet S. Gurkaynak, 2010. "How Useful Are Estimated DSGE Model Forecasts for Central Bankers?," Brookings Papers on Economic Activity, Economic Studies Program, The Brookings Institution, vol. 41(2 (Fall)), pages 209-259.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Patrick Minford & Yi Wang & Peng Zhou, 2020. "Resolving the public-sector wage premium puzzle by indirect inference," Applied Economics, Taylor & Francis Journals, vol. 52(7), pages 726-741, February.
    2. Chou, Jenyu & Easaw, Joshy & Minford, Patrick, 2023. "Does inattentiveness matter for DSGE modeling? An empirical investigation," Economic Modelling, Elsevier, vol. 118(C).
    3. Meenagh, David & Minford, Patrick & Wickens, Michael, 2021. "Estimating macro models and the potentially misleading nature of Bayesian estimation," Cardiff Economics Working Papers E2021/22, Cardiff University, Cardiff Business School, Economics Section.
    4. David Meenagh & Patrick Minford & Michael Wickens & Yongdeng Xu, 2019. "Testing DSGE Models by Indirect Inference: a Survey of Recent Findings," Open Economies Review, Springer, vol. 30(3), pages 593-620, July.
    5. Loberto, Michele & Perricone, Chiara, 2017. "Does trend inflation make a difference?," Economic Modelling, Elsevier, vol. 61(C), pages 351-375.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    2. Rossi, Barbara, 2013. "Advances in Forecasting under Instability," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1203-1324, Elsevier.
    3. Amat, Christophe & Michalski, Tomasz & Stoltz, Gilles, 2018. "Fundamentals and exchange rate forecastability with simple machine learning methods," Journal of International Money and Finance, Elsevier, vol. 88(C), pages 1-24.
    4. Raffaella Giacomini & Barbara Rossi, 2013. "Forecasting in macroeconomics," Chapters, in: Nigar Hashimzade & Michael A. Thornton (ed.), Handbook of Research Methods and Applications in Empirical Macroeconomics, chapter 17, pages 381-408, Edward Elgar Publishing.
    5. Feng, Wenjun & Zhang, Zhengjun, 2023. "Currency exchange rate predictability: The new power of Bitcoin prices," Journal of International Money and Finance, Elsevier, vol. 132(C).
    6. Barbara Rossi, 2013. "Exchange Rate Predictability," Journal of Economic Literature, American Economic Association, vol. 51(4), pages 1063-1119, December.
    7. Joseph P. Byrne & Dimitris Korobilis & Pinho J. Ribeiro, 2018. "On The Sources Of Uncertainty In Exchange Rate Predictability," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 59(1), pages 329-357, February.
    8. Barbara Rossi, 2019. "Forecasting in the presence of instabilities: How do we know whether models predict well and how to improve them," Economics Working Papers 1711, Department of Economics and Business, Universitat Pompeu Fabra, revised Jul 2021.
    9. Faust, Jon & Wright, Jonathan H., 2013. "Forecasting Inflation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 2-56, Elsevier.
    10. Byrne, Joseph P. & Korobilis, Dimitris & Ribeiro, Pinho J., 2016. "Exchange rate predictability in a changing world," Journal of International Money and Finance, Elsevier, vol. 62(C), pages 1-24.
    11. Rossi, Barbara & Gürkaynak, Refet & Kısacıkoğlu, Burçin, 2013. "Do DSGE Models Forecast More Accurately Out-of-Sample than VAR Models?," CEPR Discussion Papers 9576, C.E.P.R. Discussion Papers.
    12. Ciner, Cetin, 2017. "Predicting white metal prices by a commodity sensitive exchange rate," International Review of Financial Analysis, Elsevier, vol. 52(C), pages 309-315.
    13. Byrne, Joseph P. & Korobilis, Dimitris & Ribeiro, Pinho J., 2014. "On the Sources of Uncertainty in Exchange Rate Predictability," 2007 Annual Meeting, July 29-August 1, 2007, Portland, Oregon TN 2015-24, American Agricultural Economics Association (New Name 2008: Agricultural and Applied Economics Association).
    14. Ayse Kabukcuoglu & Enrique Martínez-García, 2016. "What Helps Forecast U.S. Inflation?—Mind the Gap!," Koç University-TUSIAD Economic Research Forum Working Papers 1615, Koc University-TUSIAD Economic Research Forum.
    15. Liu, Chunping & Minford, Patrick, 2014. "Comparing behavioural and rational expectations for the US post-war economy," Economic Modelling, Elsevier, vol. 43(C), pages 407-415.
    16. Martinez-Martin Jaime & Morris Richard & Onorante Luca & Piersanti Fabio Massimo, 2024. "Merging Structural and Reduced-Form Models for Forecasting," The B.E. Journal of Macroeconomics, De Gruyter, vol. 24(1), pages 399-437, January.
    17. Tanya Molodtsova & Alex Nikolsko-Rzhevskyy & David H. Papell, 2011. "Taylor Rules and the Euro," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 43, pages 535-552, March.
    18. Wang, Rudan & Morley, Bruce & Stamatogiannis, Michalis P., 2019. "Forecasting the exchange rate using nonlinear Taylor rule based models," International Journal of Forecasting, Elsevier, vol. 35(2), pages 429-442.
    19. Rossi, Barbara & Sekhposyan, Tatevik, 2011. "Understanding models' forecasting performance," Journal of Econometrics, Elsevier, vol. 164(1), pages 158-172, September.
    20. Kenneth S. Rogoff & Vania Stavrakeva, 2008. "The Continuing Puzzle of Short Horizon Exchange Rate Forecasting," NBER Working Papers 14071, National Bureau of Economic Research, Inc.

    More about this item

    Keywords

    Out of sample forecasts; DSGE; VAR; Specification tests; Indirect inference; Forecast performance; E10; E17;
    All these keywords.

    JEL classification:

    • E10 - Macroeconomics and Monetary Economics - - General Aggregative Models - - - General
    • E17 - Macroeconomics and Monetary Economics - - General Aggregative Models - - - Forecasting and Simulation: Models and Applications

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:italej:v:1:y:2015:i:3:p:333-351. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.