IDEAS home Printed from https://ideas.repec.org/a/eee/econom/v170y2012i1p1-14.html
   My bibliography  Save this article

In-sample tests of predictive ability: A new approach

Author

Listed:
  • Clark, Todd E.
  • McCracken, Michael W.

Abstract

This paper presents evidence linking in-sample tests of predictive content and out-of-sample forecast accuracy. Our approach focuses on the negative effect that finite-sample estimation error has on forecast accuracy despite the presence of significant population-level predictive content. We derive in-sample tests that assess whether a variable has predictive content and whether this content is estimated precisely enough to improve forecast accuracy. Our tests are asymptotically non-central chi-square or non-central normal. We provide a convenient bootstrap for computing critical values. In Monte Carlo and empirical analysis, we examine the effectiveness of our testing procedure.

Suggested Citation

  • Clark, Todd E. & McCracken, Michael W., 2012. "In-sample tests of predictive ability: A new approach," Journal of Econometrics, Elsevier, vol. 170(1), pages 1-14.
    • Handle: RePEc:eee:econom:v:170:y:2012:i:1:p:1-14
    DOI: 10.1016/j.jeconom.2010.09.012
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S030440761200111X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jeconom.2010.09.012?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Ivo Welch & Amit Goyal, 2008. "A Comprehensive Look at The Empirical Performance of Equity Premium Prediction," The Review of Financial Studies, Society for Financial Studies, vol. 21(4), pages 1455-1508, July.
    2. Kilian, Lutz, 1999. "Exchange Rates and Monetary Fundamentals: What Do We Learn from Long-Horizon Regressions?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 14(5), pages 491-510, Sept.-Oct.
    3. Clark, Todd E. & McCracken, Michael W., 2015. "Nested forecast model comparisons: A new approach to testing equal accuracy," Journal of Econometrics, Elsevier, vol. 186(1), pages 160-177.
    4. Andrew Ang & Geert Bekaert, 2007. "Stock Return Predictability: Is it There?," The Review of Financial Studies, Society for Financial Studies, vol. 20(3), pages 651-707.
    5. Torous, Walter & Valkanov, Rossen, 2000. "Boundaries of Predictability: Noisy Predictive Regressions," University of California at Los Angeles, Anderson Graduate School of Management qt33p7672z, Anderson Graduate School of Management, UCLA.
    6. Clark, Todd E. & McCracken, Michael W., 2005. "The power of tests of predictive ability in the presence of structural breaks," Journal of Econometrics, Elsevier, vol. 124(1), pages 1-31, January.
    7. Kirby, Chris, 1997. "Measuring the Predictable Variation in Stock and Bond Returns," The Review of Financial Studies, Society for Financial Studies, vol. 10(3), pages 579-630.
    8. Hodrick, Robert J, 1992. "Dividend Yields and Expected Stock Returns: Alternative Procedures for Inference and Measurement," The Review of Financial Studies, Society for Financial Studies, vol. 5(3), pages 357-386.
    9. Inoue, Atsushi & Kilian, Lutz, 2006. "On the selection of forecasting models," Journal of Econometrics, Elsevier, vol. 130(2), pages 273-306, February.
    10. John Y. Campbell & Samuel B. Thompson, 2008. "Predicting Excess Stock Returns Out of Sample: Can Anything Beat the Historical Average?," The Review of Financial Studies, Society for Financial Studies, vol. 21(4), pages 1509-1531, July.
    11. White, Halbert, 1980. "A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity," Econometrica, Econometric Society, vol. 48(4), pages 817-838, May.
    12. Nelson, Charles R & Kim, Myung J, 1993. "Predictable Stock Returns: The Role of Small Sample Bias," Journal of Finance, American Finance Association, vol. 48(2), pages 641-661, June.
    13. Silvia Goncalves & Lutz Kilian, 2007. "Asymptotic and Bootstrap Inference for AR(∞) Processes with Conditional Heteroskedasticity," Econometric Reviews, Taylor & Francis Journals, vol. 26(6), pages 609-641.
    14. Rapach, David E. & Wohar, Mark E., 2006. "In-sample vs. out-of-sample tests of stock return predictability in the context of data mining," Journal of Empirical Finance, Elsevier, vol. 13(2), pages 231-247, March.
    15. Pesaran, M Hashem & Timmermann, Allan, 1995. "Predictability of Stock Returns: Robustness and Economic Significance," Journal of Finance, American Finance Association, vol. 50(4), pages 1201-1228, September.
    16. Newey, Whitney & West, Kenneth, 2014. "A simple, positive semi-definite, heteroscedasticity and autocorrelation consistent covariance matrix," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 33(1), pages 125-132.
    17. Hansen, Bruce E, 1996. "Erratum: The Likelihood Ratio Test under Nonstandard Conditions: Testing the Markov Switching Model of GNP," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 11(2), pages 195-198, March-Apr.
    18. Kilian, Lutz & Inoue, Atsushi, 2004. "Bagging Time Series Models," CEPR Discussion Papers 4333, C.E.P.R. Discussion Papers.
    19. de Jong, Robert M., 1997. "Central Limit Theorems for Dependent Heterogeneous Random Variables," Econometric Theory, Cambridge University Press, vol. 13(3), pages 353-367, June.
    20. Halbert White, 2000. "A Reality Check for Data Snooping," Econometrica, Econometric Society, vol. 68(5), pages 1097-1126, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Burcu Erik & Marco J. Lombardi & Dubravko Mihaljek & Hyun Song Shin, 2020. "The Dollar, Bank Leverage, and Real Economic Activity: An Evolving Relationship," AEA Papers and Proceedings, American Economic Association, vol. 110, pages 529-534, May.
    2. Hillebrand, Eric & Lukas, Manuel & Wei, Wei, 2021. "Bagging weak predictors," International Journal of Forecasting, Elsevier, vol. 37(1), pages 237-254.
    3. Tom Boot & Andreas Pick, 2017. "A near optimal test for structural breaks when forecasting under square error loss," Tinbergen Institute Discussion Papers 17-039/III, Tinbergen Institute.
    4. Hossein Hassani & Emmanuel Sirimal Silva, 2015. "A Kolmogorov-Smirnov Based Test for Comparing the Predictive Accuracy of Two Sets of Forecasts," Econometrics, MDPI, vol. 3(3), pages 1-20, August.
    5. Juan Antolin-Diaz & Thomas Drechsel & Ivan Petrella, 2017. "Tracking the Slowdown in Long-Run GDP Growth," The Review of Economics and Statistics, MIT Press, vol. 99(2), pages 343-356, May.
    6. Lutz Kilian & Robert J. Vigfusson, 2013. "Do Oil Prices Help Forecast U.S. Real GDP? The Role of Nonlinearities and Asymmetries," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 31(1), pages 78-93, January.
    7. Petrella, Ivan & Drechsel, Thomas & Antolin-Diaz, Juan, 2014. "Following the Trend: Tracking GDP when Long-Run Growth is Uncertain," CEPR Discussion Papers 10272, C.E.P.R. Discussion Papers.
    8. Pincheira-Brown, Pablo & Selaive, Jorge & Nolazco, Jose Luis, 2019. "Forecasting inflation in Latin America with core measures," International Journal of Forecasting, Elsevier, vol. 35(3), pages 1060-1071.
    9. Su, Hao & Ying, Chengwei & Zhu, Xiaoneng, 2022. "Disaster risk matters in the bond market," Finance Research Letters, Elsevier, vol. 47(PA).
    10. Chauvet, Marcelle & Senyuz, Zeynep & Yoldas, Emre, 2015. "What does financial volatility tell us about macroeconomic fluctuations?," Journal of Economic Dynamics and Control, Elsevier, vol. 52(C), pages 340-360.
    11. Burcu Erik & Marco Jacopo Lombardi & Dubravko Mihaljek & Hyun Song Shin, 2019. "Financial conditions and purchasing managers' indices: exploring the links," BIS Quarterly Review, Bank for International Settlements, September.
    12. Calhoun, Gray, 2014. "Out-Of-Sample Comparisons of Overfit Models," Staff General Research Papers Archive 32462, Iowa State University, Department of Economics.
    13. Hsiu-Hsin Ko, 2016. "Exchange Rate Predictability in Finite Samples," The Japanese Economic Review, Springer, vol. 67(3), pages 361-378, September.
    14. Hsiu-Hsin Ko, 2016. "Exchange Rate Predictability in Finite Samples," The Japanese Economic Review, Japanese Economic Association, vol. 67(3), pages 361-378, September.
    15. Boot, Tom & Pick, Andreas, 2020. "Does modeling a structural break improve forecast accuracy?," Journal of Econometrics, Elsevier, vol. 215(1), pages 35-59.
    16. Mohan Subbiah & Frank J Fabozzi, 2016. "Equity style allocation: A nonparametric approach," Journal of Asset Management, Palgrave Macmillan, vol. 17(3), pages 141-164, May.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Clark, Todd E. & McCracken, Michael W., 2015. "Nested forecast model comparisons: A new approach to testing equal accuracy," Journal of Econometrics, Elsevier, vol. 186(1), pages 160-177.
    2. Chen, Long, 2009. "On the reversal of return and dividend growth predictability: A tale of two periods," Journal of Financial Economics, Elsevier, vol. 92(1), pages 128-151, April.
    3. Clark, Todd & McCracken, Michael, 2013. "Advances in Forecast Evaluation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1107-1201, Elsevier.
    4. Maio, Paulo & Santa-Clara, Pedro, 2012. "Multifactor models and their consistency with the ICAPM," Journal of Financial Economics, Elsevier, vol. 106(3), pages 586-613.
    5. Rapach, David & Zhou, Guofu, 2013. "Forecasting Stock Returns," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 328-383, Elsevier.
    6. Boudoukh, Jacob & Israel, Ronen & Richardson, Matthew, 2022. "Biases in long-horizon predictive regressions," Journal of Financial Economics, Elsevier, vol. 145(3), pages 937-969.
    7. Atanasov, Victoria, 2018. "World output gap and global stock returns," Journal of Empirical Finance, Elsevier, vol. 48(C), pages 181-197.
    8. Favero, Carlo A. & Gozluklu, Arie E. & Tamoni, Andrea, 2011. "Demographic Trends, the Dividend-Price Ratio, and the Predictability of Long-Run Stock Market Returns," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 46(5), pages 1493-1520, October.
    9. Rapach, David E. & Wohar, Mark E., 2006. "In-sample vs. out-of-sample tests of stock return predictability in the context of data mining," Journal of Empirical Finance, Elsevier, vol. 13(2), pages 231-247, March.
    10. Hai Lin & Daniel Quill & Henk Berkman, 2016. "Information diffusion and the predictability of New Zealand stock market returns," Accounting and Finance, Accounting and Finance Association of Australia and New Zealand, vol. 56(3), pages 749-785, September.
    11. Jurdi, Doureige J., 2022. "Predicting the Australian equity risk premium," Pacific-Basin Finance Journal, Elsevier, vol. 71(C).
    12. Todd E. Clark & Michael W. McCracken, 2010. "Reality checks and nested forecast model comparisons," Working Papers 2010-032, Federal Reserve Bank of St. Louis.
    13. Neil Kellard & John Nankervis & Fotis Papadimitriou, 2007. "Predicting the UK Equity Premium with Dividend Ratios: An Out-Of-Sample Recursive Residuals Graphical Approach," Money Macro and Finance (MMF) Research Group Conference 2006 129, Money Macro and Finance Research Group.
    14. Rapach, David E. & Wohar, Mark E. & Rangvid, Jesper, 2005. "Macro variables and international stock return predictability," International Journal of Forecasting, Elsevier, vol. 21(1), pages 137-166.
    15. Stephan Jank, 2015. "Changes in the Composition of Publicly Traded Firms: Implications for the Dividend-Price Ratio and Return Predictability," Management Science, INFORMS, vol. 61(6), pages 1362-1377, June.
    16. Galvão, Ana Beatriz, 2013. "Changes in predictive ability with mixed frequency data," International Journal of Forecasting, Elsevier, vol. 29(3), pages 395-410.
    17. Rossi, Barbara, 2013. "Advances in Forecasting under Instability," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1203-1324, Elsevier.
    18. repec:grz:wpaper:2012-02 is not listed on IDEAS
    19. Dladla, Pholile & Malikane, Christopher, 2019. "Stock return predictability: Evidence from a structural model," International Review of Economics & Finance, Elsevier, vol. 59(C), pages 412-424.
    20. Ivo Welch & Amit Goyal, 2008. "A Comprehensive Look at The Empirical Performance of Equity Premium Prediction," The Review of Financial Studies, Society for Financial Studies, vol. 21(4), pages 1455-1508, July.
    21. Dichtl, Hubert & Drobetz, Wolfgang & Neuhierl, Andreas & Wendt, Viktoria-Sophie, 2021. "Data snooping in equity premium prediction," International Journal of Forecasting, Elsevier, vol. 37(1), pages 72-94.

    More about this item

    Keywords

    Predictability; Forecast accuracy; In-sample;
    All these keywords.

    JEL classification:

    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:econom:v:170:y:2012:i:1:p:1-14. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jeconom .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.