IDEAS home Printed from https://ideas.repec.org/a/eee/econom/v198y2017i2p231-252.html
   My bibliography  Save this article

Tests of equal accuracy for nested models with estimated factors

Author

Listed:
  • Gonçalves, Sílvia
  • McCracken, Michael W.
  • Perron, Benoit

Abstract

In this paper we develop asymptotics for tests of equal predictive ability between nested models when factor-augmented regressions are used to forecast. We provide conditions under which the estimation of the factors does not affect the asymptotic distributions developed in Clark and McCracken (2001) and McCracken (2007). This enables researchers to use the existing tabulated critical values when conducting inference despite the presence of estimated predictors. As an intermediate result, we derive the asymptotic properties of the principal components estimator over recursive windows. We provide simulation evidence on the finite sample effects of factor estimation and apply the tests to the case of forecasting excess returns to the S&P 500 Composite Index.

Suggested Citation

  • Gonçalves, Sílvia & McCracken, Michael W. & Perron, Benoit, 2017. "Tests of equal accuracy for nested models with estimated factors," Journal of Econometrics, Elsevier, vol. 198(2), pages 231-252.
    • Handle: RePEc:eee:econom:v:198:y:2017:i:2:p:231-252
    DOI: 10.1016/j.jeconom.2017.01.004
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0304407617300180
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jeconom.2017.01.004?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Jianqing Fan & Yuan Liao & Martina Mincheva, 2013. "Large covariance estimation by thresholding principal orthogonal complements," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 75(4), pages 603-680, September.
    2. Gonçalves, Sílvia & Perron, Benoit, 2014. "Bootstrapping factor-augmented regression models," Journal of Econometrics, Elsevier, vol. 182(1), pages 156-173.
    3. Clark, Todd E. & McCracken, Michael W., 2001. "Tests of equal forecast accuracy and encompassing for nested models," Journal of Econometrics, Elsevier, vol. 105(1), pages 85-110, November.
    4. West, Kenneth D, 1996. "Asymptotic Inference about Predictive Ability," Econometrica, Econometric Society, vol. 64(5), pages 1067-1084, September.
    5. Michael W. McCracken & Serena Ng, 2016. "FRED-MD: A Monthly Database for Macroeconomic Research," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(4), pages 574-589, October.
    6. Shintani, Mototsugu, 2005. "Nonlinear Forecasting Analysis Using Diffusion Indexes: An Application to Japan," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 37(3), pages 517-538, June.
    7. Todd Clark & Michael McCracken, 2012. "Reality Checks and Comparisons of Nested Predictive Models," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 30(1), pages 53-66.
    8. Todd Clark & Michael McCracken, 2005. "Evaluating Direct Multistep Forecasts," Econometric Reviews, Taylor & Francis Journals, vol. 24(4), pages 369-404.
    9. Peter Reinhard Hansen & Allan Timmermann, 2015. "Equivalence Between Out‐of‐Sample Forecast Comparisons and Wald Statistics," Econometrica, Econometric Society, vol. 83, pages 2485-2505, November.
    10. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    11. Matteo Ciccarelli & Benoît Mojon, 2010. "Global Inflation," The Review of Economics and Statistics, MIT Press, vol. 92(3), pages 524-535, August.
    12. Elena Andreou & Eric Ghysels & Andros Kourtellos, 2013. "Should Macroeconomic Forecasters Use Daily Financial Data and How?," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 31(2), pages 240-251, April.
    13. James H. Stock & Mark W.Watson, 2003. "Forecasting Output and Inflation: The Role of Asset Prices," Journal of Economic Literature, American Economic Association, vol. 41(3), pages 788-829, September.
    14. Cheng, Xu & Hansen, Bruce E., 2015. "Forecasting with factor-augmented regression: A frequentist model averaging approach," Journal of Econometrics, Elsevier, vol. 186(2), pages 280-293.
    15. Jack Fosten, 2016. "Forecast evaluation with factor-augmented models," University of East Anglia School of Economics Working Paper Series 2016-05, School of Economics, University of East Anglia, Norwich, UK..
    16. Çakmaklı, Cem & van Dijk, Dick, 2016. "Getting the most out of macroeconomic information for predicting excess stock returns," International Journal of Forecasting, Elsevier, vol. 32(3), pages 650-668.
    17. Jushan Bai, 2003. "Inferential Theory for Factor Models of Large Dimensions," Econometrica, Econometric Society, vol. 71(1), pages 135-171, January.
    18. Jushan Bai & Serena Ng, 2006. "Confidence Intervals for Diffusion Index Forecasts and Inference for Factor-Augmented Regressions," Econometrica, Econometric Society, vol. 74(4), pages 1133-1150, July.
    19. Han, Xu & Inoue, Atsushi, 2015. "Tests For Parameter Instability In Dynamic Factor Models," Econometric Theory, Cambridge University Press, vol. 31(5), pages 1117-1152, October.
    20. Murphy, Kevin M & Topel, Robert H, 2002. "Estimation and Inference in Two-Step Econometric Models," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 88-97, January.
    21. Bai, Jushan & Ng, Serena, 2013. "Principal components estimation and identification of static factors," Journal of Econometrics, Elsevier, vol. 176(1), pages 18-29.
    22. Adrian Pagan, 1986. "Two Stage and Related Estimators and Their Applications," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 53(4), pages 517-538.
    23. Hjalmarsson, Erik, 2010. "Predicting Global Stock Returns," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 45(1), pages 49-80, February.
    24. Sydney C. Ludvigson & Serena Ng, 2009. "A Factor Analysis of Bond Risk Premia," NBER Working Papers 15188, National Bureau of Economic Research, Inc.
    25. Hofmann, Boris, 2009. "Do monetary indicators lead euro area inflation?," Journal of International Money and Finance, Elsevier, vol. 28(7), pages 1165-1181, November.
    26. Chamberlain, Gary & Rothschild, Michael, 1983. "Arbitrage, Factor Structure, and Mean-Variance Analysis on Large Asset Markets," Econometrica, Econometric Society, vol. 51(5), pages 1281-1304, September.
    27. Stock, James H & Watson, Mark W, 1996. "Evidence on Structural Instability in Macroeconomic Time Series Relations," Journal of Business & Economic Statistics, American Statistical Association, vol. 14(1), pages 11-30, January.
    28. Stock, James H & Watson, Mark W, 2002. "Macroeconomic Forecasting Using Diffusion Indexes," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(2), pages 147-162, April.
    29. McCracken, Michael W., 2007. "Asymptotics for out of sample tests of Granger causality," Journal of Econometrics, Elsevier, vol. 140(2), pages 719-752, October.
    30. Jennie Bai, 2010. "Equity premium predictions with adaptive macro indexes," Staff Reports 475, Federal Reserve Bank of New York.
    31. Cem Cakmakli & Dick van Dijk, 2010. "Getting the Most out of Macroeconomic Information for Predicting Stock Returns and Volatility," Tinbergen Institute Discussion Papers 10-115/4, Tinbergen Institute.
    32. Jushan Bai, 2009. "Panel Data Models With Interactive Fixed Effects," Econometrica, Econometric Society, vol. 77(4), pages 1229-1279, July.
    33. Pagan, Adrian, 1984. "Econometric Issues in the Analysis of Regressions with Generated Regressors," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 25(1), pages 221-247, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Massacci, Daniele & Kapetanios, George, 2024. "Forecasting in factor augmented regressions under structural change," International Journal of Forecasting, Elsevier, vol. 40(1), pages 62-76.
    2. Giovannelli, Alessandro & Massacci, Daniele & Soccorsi, Stefano, 2021. "Forecasting stock returns with large dimensional factor models," Journal of Empirical Finance, Elsevier, vol. 63(C), pages 252-269.
    3. Arabinda Basistha, 2023. "Estimation of short‐run predictive factor for US growth using state employment data," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 42(1), pages 34-50, January.
    4. Luiz Renato Lima & Lucas Lúcio Godeiro & Mohammed Mohsin, 2021. "Time-Varying Dictionary and the Predictive Power of FED Minutes," Computational Economics, Springer;Society for Computational Economics, vol. 57(1), pages 149-181, January.
    5. Marine Carrasco & Barbara Rossi, 2016. "In-Sample Inference and Forecasting in Misspecified Factor Models," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(3), pages 313-338, July.
    6. Andreou, Elena & Ghysels, Eric, 2021. "Predicting the VIX and the volatility risk premium: The role of short-run funding spreads Volatility Factors," Journal of Econometrics, Elsevier, vol. 220(2), pages 366-398.
    7. Arabinda Basistha & Richard Startz, 2024. "Measuring persistent global economic factors with output, commodity price, and commodity currency data," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 43(7), pages 2860-2885, November.
    8. Daniel Borup & Bent Jesper Christensen & Yunus Emre Ergemen, 2019. "Assessing predictive accuracy in panel data models with long-range dependence," CREATES Research Papers 2019-04, Department of Economics and Business Economics, Aarhus University.
    9. Fosten, Jack, 2017. "Confidence intervals in regressions with estimated factors and idiosyncratic components," Economics Letters, Elsevier, vol. 157(C), pages 71-74.
    10. Corradi, Valentina & Fosten, Jack & Gutknecht, Daniel, 2024. "Predictive ability tests with possibly overlapping models," Journal of Econometrics, Elsevier, vol. 241(1).
    11. Antoine A. Djogbenou, 2021. "Model selection in factor-augmented regressions with estimated factors," Econometric Reviews, Taylor & Francis Journals, vol. 40(5), pages 470-503, April.
    12. Daniel Borup & Jonas N. Eriksen & Mads M. Kjær & Martin Thyrsgaard, 2024. "Predicting Bond Return Predictability," Management Science, INFORMS, vol. 70(2), pages 931-951, February.
    13. Arabinda Basistha & Richard Startz, 2024. "Measuring persistent global economic factors with output, commodity price, and commodity currency data," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 43(7), pages 2860-2885, November.
    14. Luiz Renato Lima & Lucas Lúcio Godeiro, 2023. "Equity‐premium prediction: Attention is all you need," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 38(1), pages 105-122, January.
    15. Karen Miranda & Pilar Poncela & Esther Ruiz, 2022. "Dynamic factor models: Does the specification matter?," SERIEs: Journal of the Spanish Economic Association, Springer;Spanish Economic Association, vol. 13(1), pages 397-428, May.
    16. In Choi & Hanbat Jeong, 2020. "Differencing versus nondifferencing in factor‐based forecasting," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(6), pages 728-750, September.
    17. Daniel Borup & Martin Thyrsgaard, 2017. "Statistical tests for equal predictive ability across multiple forecasting methods," CREATES Research Papers 2017-19, Department of Economics and Business Economics, Aarhus University.
    18. Michael W. McCracken, 2020. "Tests of Conditional Predictive Ability: Existence, Size, and Power," Working Papers 2020-050, Federal Reserve Bank of St. Louis.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Stock, J.H. & Watson, M.W., 2016. "Dynamic Factor Models, Factor-Augmented Vector Autoregressions, and Structural Vector Autoregressions in Macroeconomics," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 415-525, Elsevier.
    2. Jack Fosten, 2016. "Forecast evaluation with factor-augmented models," University of East Anglia School of Economics Working Paper Series 2016-05, School of Economics, University of East Anglia, Norwich, UK..
    3. Varlam Kutateladze, 2021. "The Kernel Trick for Nonlinear Factor Modeling," Papers 2103.01266, arXiv.org.
    4. Kutateladze, Varlam, 2022. "The kernel trick for nonlinear factor modeling," International Journal of Forecasting, Elsevier, vol. 38(1), pages 165-177.
    5. Clark, Todd E. & West, Kenneth D., 2007. "Approximately normal tests for equal predictive accuracy in nested models," Journal of Econometrics, Elsevier, vol. 138(1), pages 291-311, May.
    6. Firmin Doko Tchatoka & Qazi Haque, 2023. "On bootstrapping tests of equal forecast accuracy for nested models," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 42(7), pages 1844-1864, November.
    7. Clark, Todd & McCracken, Michael, 2013. "Advances in Forecast Evaluation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1107-1201, Elsevier.
    8. Yoshimasa Uematsu & Takashi Yamagata, 2019. "Estimation of Weak Factor Models," ISER Discussion Paper 1053r, Institute of Social and Economic Research, Osaka University, revised Mar 2020.
    9. Karen Miranda & Pilar Poncela & Esther Ruiz, 2022. "Dynamic factor models: Does the specification matter?," SERIEs: Journal of the Spanish Economic Association, Springer;Spanish Economic Association, vol. 13(1), pages 397-428, May.
    10. In Choi & Hanbat Jeong, 2020. "Differencing versus nondifferencing in factor‐based forecasting," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(6), pages 728-750, September.
    11. Chen, Qitong & Hong, Yongmiao & Li, Haiqi, 2024. "Time-varying forecast combination for factor-augmented regressions with smooth structural changes," Journal of Econometrics, Elsevier, vol. 240(1).
    12. Corradi, Valentina & Fosten, Jack & Gutknecht, Daniel, 2024. "Predictive ability tests with possibly overlapping models," Journal of Econometrics, Elsevier, vol. 241(1).
    13. Kenneth S. Rogoff & Vania Stavrakeva, 2008. "The Continuing Puzzle of Short Horizon Exchange Rate Forecasting," NBER Working Papers 14071, National Bureau of Economic Research, Inc.
    14. Clark, Todd E. & McCracken, Michael W., 2015. "Nested forecast model comparisons: A new approach to testing equal accuracy," Journal of Econometrics, Elsevier, vol. 186(1), pages 160-177.
    15. Marine Carrasco & Barbara Rossi, 2016. "In-Sample Inference and Forecasting in Misspecified Factor Models," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(3), pages 313-338, July.
    16. Daniel Borup & Jonas N. Eriksen & Mads M. Kjær & Martin Thyrsgaard, 2024. "Predicting Bond Return Predictability," Management Science, INFORMS, vol. 70(2), pages 931-951, February.
    17. Todd E. Clark & Michael W. McCracken, 2009. "Combining Forecasts from Nested Models," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 71(3), pages 303-329, June.
    18. Yoshimasa Uematsu & Takashi Yamagata, 2019. "Estimation of Weak Factor Models," DSSR Discussion Papers 96, Graduate School of Economics and Management, Tohoku University.
    19. Yoshimasa Uematsu & Takashi Yamagata, 2020. "Inference in Weak Factor Models," ISER Discussion Paper 1080, Institute of Social and Economic Research, Osaka University.
    20. Bai, Jushan & Ng, Serena, 2019. "Rank regularized estimation of approximate factor models," Journal of Econometrics, Elsevier, vol. 212(1), pages 78-96.

    More about this item

    Keywords

    Factor model; Out-of-sample forecasts; Recursive estimation;
    All these keywords.

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C32 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes; State Space Models
    • C38 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Classification Methdos; Cluster Analysis; Principal Components; Factor Analysis
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:econom:v:198:y:2017:i:2:p:231-252. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jeconom .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.