IDEAS home Printed from https://ideas.repec.org/p/nbr/nberte/0256.html
   My bibliography  Save this paper

Encompassing Tests When No Model Is Encompassing

Author

Listed:
  • Kenneth D. West

Abstract

This paper considers regression-based tests for encompassing, when none of the models under consideration encompasses all the other models. For both in- and out-of-sample applications, I derive asymptotic distributions and propose feasible procedures to construct confidence intervals and test statistics. Procedures that are asymptotically valid under the null of encompassing (e.g., Davidson and MacKinnon (1981)) can have large asymptotic and finite sample distortions. Simulations indicate that the proposed procedures can work well in samples of size typically available, though the divergence between actual and nominal confidence interval coverage sometimes is large.

Suggested Citation

  • Kenneth D. West, 2000. "Encompassing Tests When No Model Is Encompassing," NBER Technical Working Papers 0256, National Bureau of Economic Research, Inc.
    • Handle: RePEc:nbr:nberte:0256
    Note: TWP
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/t0256.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Hansen, Lars Peter, 1982. "Large Sample Properties of Generalized Method of Moments Estimators," Econometrica, Econometric Society, vol. 50(4), pages 1029-1054, July.
    2. Davidson, Russell & MacKinnon, James G, 1981. "Several Tests for Model Specification in the Presence of Alternative Hypotheses," Econometrica, Econometric Society, vol. 49(3), pages 781-793, May.
    3. West, Kenneth D, 1996. "Asymptotic Inference about Predictive Ability," Econometrica, Econometric Society, vol. 64(5), pages 1067-1084, September.
    4. White,Halbert, 1996. "Estimation, Inference and Specification Analysis," Cambridge Books, Cambridge University Press, number 9780521574464, September.
    5. McAleer, Michael, 1995. "The significance of testing empirical non-nested models," Journal of Econometrics, Elsevier, vol. 67(1), pages 149-171, May.
    6. Yock Y. Chong & David F. Hendry, 1986. "Econometric Evaluation of Linear Macro-Economic Models," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 53(4), pages 671-690.
    7. West, Kenneth D & McCracken, Michael W, 1998. "Regression-Based Tests of Predictive Ability," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 817-840, November.
    8. Mizon, Grayham E & Richard, Jean-Francois, 1986. "The Encompassing Principle and Its Application to Testing Non-nested Hypotheses," Econometrica, Econometric Society, vol. 54(3), pages 657-678, May.
    9. Pagan, Adrian R. & Schwert, G. William, 1990. "Alternative models for conditional stock volatility," Journal of Econometrics, Elsevier, vol. 45(1-2), pages 267-290.
    10. Fair, Ray C & Shiller, Robert J, 1990. "Comparing Information in Forecasts from Econometric Models," American Economic Review, American Economic Association, vol. 80(3), pages 375-389, June.
    11. Robert F. Engle & Che-Hsiung Hong & Alex Kane, 1990. "Valuation of Variance Forecast with Simulated Option Markets," NBER Working Papers 3350, National Bureau of Economic Research, Inc.
    12. Ericsson, Neil R., 1992. "Parameter constancy, mean square forecast errors, and measuring forecast performance: An exposition, extensions, and illustration," Journal of Policy Modeling, Elsevier, vol. 14(4), pages 465-495, August.
    13. Ronald L. Cooper, 1972. "The Predictive Performance of Quarterly Econometric Models of the United States," NBER Chapters, in: Econometric Models of Cyclical Behavior, Volumes 1 and 2, pages 813-947, National Bureau of Economic Research, Inc.
    14. Ericsson, Neil R & Marquez, Jaime, 1993. "Encompassing the Forecasts of U.S. Trade Balance Models," The Review of Economics and Statistics, MIT Press, vol. 75(1), pages 19-31, February.
    15. Diebold, Francis X., 1989. "Forecast combination and encompassing: Reconciling two divergent literatures," International Journal of Forecasting, Elsevier, vol. 5(4), pages 589-592.
    16. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    17. Vuong, Quang H, 1989. "Likelihood Ratio Tests for Model Selection and Non-nested Hypotheses," Econometrica, Econometric Society, vol. 57(2), pages 307-333, March.
    18. Godfrey, L. G. & Pesaran, M. H., 1983. "Tests of non-nested regression models: Small sample adjustments and Monte Carlo evidence," Journal of Econometrics, Elsevier, vol. 21(1), pages 133-154, January.
    19. Clemen, Robert T., 1989. "Combining forecasts: A review and annotated bibliography," International Journal of Forecasting, Elsevier, vol. 5(4), pages 559-583.
    20. Godfrey, L. G., 1998. "Tests of non-nested regression models some results on small sample behaviour and the bootstrap," Journal of Econometrics, Elsevier, vol. 84(1), pages 59-74, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Giacomini, Raffaella & Komunjer, Ivana, 2005. "Evaluation and Combination of Conditional Quantile Forecasts," Journal of Business & Economic Statistics, American Statistical Association, vol. 23, pages 416-431, October.
    2. Clark, Todd E. & McCracken, Michael W., 2001. "Tests of equal forecast accuracy and encompassing for nested models," Journal of Econometrics, Elsevier, vol. 105(1), pages 85-110, November.
    3. McCracken,M.W. & West,K.D., 2001. "Inference about predictive ability," Working papers 14, Wisconsin Madison - Social Systems.
    4. Tim Robinson & Andrew Stone & Marileze van Zyl, 2003. "The Real-time Forecasting Performance of Phillips Curves," RBA Research Discussion Papers rdp2003-12, Reserve Bank of Australia.
    5. Greg Tkacz & Carolyn A. Wilkins, 2006. "Linear and Threshold Forecasts of Output and Inflation with Stock and Housing Prices," Staff Working Papers 06-25, Bank of Canada.
    6. Rosario Dell'Aquila & Elvezio Ronchetti, 2004. "Robust tests of predictive accuracy," Metron - International Journal of Statistics, Dipartimento di Statistica, Probabilità e Statistiche Applicate - University of Rome, vol. 0(2), pages 161-184.
    7. Chen, Xiaohong & Fan, Yanqin, 2006. "Estimation and model selection of semiparametric copula-based multivariate dynamic models under copula misspecification," Journal of Econometrics, Elsevier, vol. 135(1-2), pages 125-154.
    8. West, Kenneth D., 2002. "Comments on 'The state of macroeconomic forecasting'," Journal of Macroeconomics, Elsevier, vol. 24(4), pages 495-497, December.
    9. Blair, Bevan J. & Poon, Ser-Huang & Taylor, Stephen J., 2001. "Forecasting S&P 100 volatility: the incremental information content of implied volatilities and high-frequency index returns," Journal of Econometrics, Elsevier, vol. 105(1), pages 5-26, November.
    10. Greg Tkacz & Carolyn Wilkins, 2008. "Linear and threshold forecasts of output and inflation using stock and housing prices," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 27(2), pages 131-151.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. McCracken,M.W. & West,K.D., 2001. "Inference about predictive ability," Working papers 14, Wisconsin Madison - Social Systems.
    2. Massimiliano Marcellino, "undated". "Further Results on MSFE Encompassing," Working Papers 143, IGIER (Innocenzo Gasparini Institute for Economic Research), Bocconi University.
    3. Clark, Todd & McCracken, Michael, 2013. "Advances in Forecast Evaluation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1107-1201, Elsevier.
    4. Francis X. Diebold & Jose A. Lopez, 1995. "Forecast evaluation and combination," Research Paper 9525, Federal Reserve Bank of New York.
    5. Clements, Michael P. & Harvey, David I., 2011. "Combining probability forecasts," International Journal of Forecasting, Elsevier, vol. 27(2), pages 208-223.
    6. Michael P. Clements & David I. Harvey, 2010. "Forecast encompassing tests and probability forecasts," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(6), pages 1028-1062.
    7. Timo Dimitriadis & Julie Schnaitmann, 2019. "Forecast Encompassing Tests for the Expected Shortfall," Papers 1908.04569, arXiv.org, revised Aug 2020.
    8. Giacomini, Raffaella & Komunjer, Ivana, 2005. "Evaluation and Combination of Conditional Quantile Forecasts," Journal of Business & Economic Statistics, American Statistical Association, vol. 23, pages 416-431, October.
    9. Jin, Fei & Lee, Lung-fei, 2013. "Cox-type tests for competing spatial autoregressive models with spatial autoregressive disturbances," Regional Science and Urban Economics, Elsevier, vol. 43(4), pages 590-616.
    10. McCracken, Michael W., 2007. "Asymptotics for out of sample tests of Granger causality," Journal of Econometrics, Elsevier, vol. 140(2), pages 719-752, October.
    11. Han, Xiaoyi & Lee, Lung-fei, 2013. "Model selection using J-test for the spatial autoregressive model vs. the matrix exponential spatial model," Regional Science and Urban Economics, Elsevier, vol. 43(2), pages 250-271.
    12. Dimitriadis, Timo & Schnaitmann, Julie, 2021. "Forecast encompassing tests for the expected shortfall," International Journal of Forecasting, Elsevier, vol. 37(2), pages 604-621.
    13. West, Kenneth D., 2006. "Forecast Evaluation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 3, pages 99-134, Elsevier.
    14. Massimiliano Marcellino, 2008. "A linear benchmark for forecasting GDP growth and inflation?," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 27(4), pages 305-340.
    15. Tae-Hwy Lee & Yong Bao & Burak Saltoğlu, 2007. "Comparing density forecast models Previous versions of this paper have been circulated with the title, 'A Test for Density Forecast Comparison with Applications to Risk Management' since October 2003;," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 26(3), pages 203-225.
    16. Otsu, Taisuke & Whang, Yoon-Jae, 2011. "Testing For Nonnested Conditional Moment Restrictions Via Conditional Empirical Likelihood," Econometric Theory, Cambridge University Press, vol. 27(1), pages 114-153, February.
    17. Ericsson, Neil R., 2016. "Eliciting GDP forecasts from the FOMC’s minutes around the financial crisis," International Journal of Forecasting, Elsevier, vol. 32(2), pages 571-583.
    18. Ashiya, Masahiro, 2007. "Forecast accuracy of the Japanese government: Its year-ahead GDP forecast is too optimistic," Japan and the World Economy, Elsevier, vol. 19(1), pages 68-85, January.
    19. Antonis Michis, 2012. "Monitoring Forecasting Combinations with Semiparametric Regression Models," Working Papers 2012-2, Central Bank of Cyprus.
    20. Busetti, Fabio & Marcucci, Juri, 2013. "Comparing forecast accuracy: A Monte Carlo investigation," International Journal of Forecasting, Elsevier, vol. 29(1), pages 13-27.

    More about this item

    JEL classification:

    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberte:0256. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.