This file is part of IDEAS, which uses RePEc data


[ Papers | Articles | Software | Books | Chapters | Authors | Institutions | JEL Classification | NEP reports | Search | New papers by email | Author registration | Rankings | Volunteers | FAQ | Blog | Help! ]

Tests of Conditional Predictive Ability

Author info | Abstract | Publisher info | Download info | Related research | Statistics
Author Info
Raffaella Giacomini (University of California, San Diego)
Halbert White (University of California, San Diego)

Additional information is available for the following registered author(s):

Abstract

We argue that the current framework for predictive ability testing (e.g., West, 1996) is not necessarily useful for real-time forecast selection, i.e., for assessing which of two competing forecasting methods will perform better in the future. We propose an alternative framework for out-of-sample comparison of predictive ability which delivers more practically relevant conclusions. Our approach is based on inference about conditional expectations of forecasts and forecast errors rather than the unconditional expectations that are the focus of the existing literature. We capture important determinants of forecast performance that are neglected in the existing literature by evaluating what we call the forecasting method (the model and the parameter estimation procedure), rather than just the forecasting model. Compared to previous approaches, our tests are valid under more general data assumptions (heterogeneity rather than stationarity) and estimation methods, and they can handle comparison of both nested and non-nested models, which is not currently possible. To illustrate the usefulness of the proposed tests, we compare the forecast performance of three leading parameter-reduction methods for macroeconomic forecasting using a large number of predictors: a sequential model selection approach, the "diffusion indexes" approach of Stock and Watson (2002), and the use of Bayesian shrinkage estimators.

Download Info
To download:

If you experience problems downloading a file, check if you have the proper application to view it first. Information about this may be contained in the File-Format links below. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: http://repositories.cdlib.org/cgi/viewcontent.cgi?article=1012&context=ucsdecon
File Format: application/pdf
File Function:
Download Restriction: no

Publisher Info
Paper provided by Department of Economics, UC San Diego in its series University of California at San Diego, Economics Working Paper Series with number 2003-09.

Download reference. The following formats are available: HTML (with abstract), plain text (with abstract), BibTeX, RIS (EndNote, RefMan, ProCite), ReDIF
Length:
Date of creation: 29 Sep 2004
Date of revision:
Handle: RePEc:cdl:ucsdec:2003-09

Note: oai:cdlib1.org:ucsdecon-1012
Contact details of provider:
Postal: 9500 Gilman Drive, La Jolla, CA 92093-0508
Phone: (858) 534-3383
Fax: (858) 534-7040
Web page: http://repositories.cdlib.org/ucsdecon/
More information through EDIRC

For technical questions regarding this item, or to correct its listing, contact: (Christopher F. Baum).

Related research
Keywords: Forecast evaluation; asymptotic inference; parameter-reduction methods;

Other versions of this item:

This paper has been announced in the following NEP Reports: References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
  1. Clark, Todd E. & McCracken, Michael W., 2001. "Tests of equal forecast accuracy and encompassing for nested models," Journal of Econometrics, Elsevier, vol. 105(1), pages 85-110, November. [Downloadable!] (restricted)
    Other versions:
  2. Fama, Eugene F & MacBeth, James D, 1973. "Risk, Return, and Equilibrium: Empirical Tests," Journal of Political Economy, University of Chicago Press, vol. 81(3), pages 607-36, May-June. [Downloadable!] (restricted)
  3. Robert B. Litterman, 1985. "Forecasting with Bayesian vector autoregressions five years of experience," Working Papers 274, Federal Reserve Bank of Minneapolis. [Downloadable!]
    Other versions:
  4. Newey, Whitney K & West, Kenneth D, 1987. "A Simple, Positive Semi-definite, Heteroskedasticity and Autocorrelation Consistent Covariance Matrix," Econometrica, Econometric Society, vol. 55(3), pages 703-08, May. [Downloadable!] (restricted)
    Other versions:
  5. McCulloch, Robert & Rossi, Peter E., 1990. "Posterior, predictive, and utility-based approaches to testing the arbitrage pricing theory," Journal of Financial Economics, Elsevier, vol. 28(1-2), pages 7-38. [Downloadable!] (restricted)
  6. Raffaella Giacomini, 2002. "Comparing Density Forecasts via Weighted Likelihood Ratio Tests: Asymptotic and Bootstrap Methods," Boston College Working Papers in Economics 583, Boston College Department of Economics. [Downloadable!]
  7. Raffaella Giacomini & Ivana Komunjer, 2003. "Evaluation and Combination of Conditional Quantile Forecasts," Boston College Working Papers in Economics 571, Boston College Department of Economics. [Downloadable!]
    Other versions:
  8. Francis X. Diebold & Jose A. Lopez, 1996. "Forecast Evaluation and Combination," NBER Technical Working Papers 0192, National Bureau of Economic Research, Inc. [Downloadable!] (restricted)
    Other versions:
  9. Andrews, Donald W K, 1991. "Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimation," Econometrica, Econometric Society, vol. 59(3), pages 817-58, May. [Downloadable!] (restricted)
    Other versions:
  10. Kevin D. Hoover & Stephen J. Perez, 1999. "Data mining reconsidered: encompassing and the general-to-specific approach to specification search," Econometrics Journal, Royal Economic Society, vol. 2(2), pages 167-191.
    Other versions:
  11. West, Kenneth D, 1996. "Asymptotic Inference about Predictive Ability," Econometrica, Econometric Society, vol. 64(5), pages 1067-84, September. [Downloadable!] (restricted)
    Other versions:
  12. Inoue, Atsushi & Kilian, Lutz, 2002. "In-Sample or Out-of-Sample Tests of Predictability: Which One Should We Use?," CEPR Discussion Papers 3671, C.E.P.R. Discussion Papers. [Downloadable!] (restricted)
    Other versions:
  13. West, Kenneth D. & Edison, Hali J. & Cho, Dongchul, 1993. "A utility-based comparison of some models of exchange rate volatility," Journal of International Economics, Elsevier, vol. 35(1-2), pages 23-45, August. [Downloadable!] (restricted)
    Other versions:
  14. Stock, James H & Watson, Mark W, 2002. "Macroeconomic Forecasting Using Diffusion Indexes," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(2), pages 147-62, April.
  15. Neil R. Ericsson, 2000. "Predictable uncertainty in economic forecasting," International Finance Discussion Papers 695, Board of Governors of the Federal Reserve System (U.S.). [Downloadable!]
  16. Lopez, Jose A, 2001. "Evaluating the Predictive Accuracy of Volatility Models," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 20(2), pages 87-109, March.
    Other versions:
  17. Halbert White, 2000. "A Reality Check for Data Snooping," Econometrica, Econometric Society, vol. 68(5), pages 1097-1126, September.
  18. Fisher, Franklin M, 1970. "Tests of Equality Between Sets of Coefficients in Two Linear Regressions: An Expository Note," Econometrica, Econometric Society, vol. 38(2), pages 361-66, March. [Downloadable!] (restricted)
  19. Corradi, Valentina & Swanson, Norman R. & Olivetti, Claudia, 2001. "Predictive ability with cointegrated variables," Journal of Econometrics, Elsevier, vol. 104(2), pages 315-358, September. [Downloadable!] (restricted)
  20. West, Kenneth D & McCracken, Michael W, 1998. "Regression-Based Tests of Predictive Ability," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 817-40, November.
    Other versions:
  21. Sims, Christopher A & Zha, Tao, 1998. "Bayesian Methods for Dynamic Multivariate Models," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 949-68, November.
    Other versions:
  22. Norman R. Swanson, 2000. "An Out of Sample Test for Granger Causality," Econometric Society World Congress 2000 Contributed Papers 0362, Econometric Society. [Downloadable!]
  23. White, Halbert & Domowitz, Ian, 1984. "Nonlinear Regression with Dependent Observations," Econometrica, Econometric Society, vol. 52(1), pages 143-61, January. [Downloadable!] (restricted)
  24. Mc Cracken, Michael W., 2000. "Robust out-of-sample inference," Journal of Econometrics, Elsevier, vol. 99(2), pages 195-223, December. [Downloadable!] (restricted)
  25. Michael P. Clements & David F. Hendry, 2001. "Forecasting Non-Stationary Economic Time Series," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262531895, January.
  26. Donald W. K. Andrews, 2002. "Higher-Order Improvements of a Computationally Attractive "k"-Step Bootstrap for Extremum Estimators," Econometrica, Econometric Society, vol. 70(1), pages 119-162, January. [Downloadable!] (restricted)
    Other versions:
  27. Raffaella Giacomini, 2002. "Comparing Density Forecasts via Weighted Likelihood Ratio Tests: Asymptotic and Bootstrap Methods," University of California at San Diego, Economics Working Paper Series 2002-12, Department of Economics, UC San Diego. [Downloadable!]
  28. Harvey, David & Leybourne, Stephen & Newbold, Paul, 1997. "Testing the equality of prediction mean squared errors," International Journal of Forecasting, Elsevier, vol. 13(2), pages 281-291, June. [Downloadable!] (restricted)
  29. Diebold, Francis X & Mariano, Roberto S, 1995. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 13(3), pages 253-63, July.
    Other versions:
  30. Bierens, Herman J, 1990. "A Consistent Conditional Moment Test of Functional Form," Econometrica, Econometric Society, vol. 58(6), pages 1443-58, November. [Downloadable!] (restricted)
    Other versions:
Full references

Cited by:
(explanations, Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.)
This item has more than 25 citations. To prevent cluttering this page, these citations are listed on a separate page.
Statistics
Access and download statistics

Did you know? IDEAS is not the only service displaying RePEc data. Choose on RePEc which service fits your needs best.

This page was last updated on 2009-10-21.

This information is provided to you by IDEAS at the Department of Economics, College of Liberal Arts and Sciences, University of Connecticut using RePEc data on a server sponsored by the Society for Economic Dynamics.