IDEAS home Printed from https://ideas.repec.org/p/ecl/corcae/06-09.html
   My bibliography  Save this paper

Robust Model Selection in Dynamic Models with an Application to Comparing Predictive Accuracy

Author

Listed:
  • Choi, Hwan-sik

    (Cornell U)

  • Kiefer, Nicholas M.

    (Cornell U)

Abstract

A model selection procedure based on a general criterion function, with an example of the Kullback-Leibler Information Criterion (KLIC) using quasi-likelihood functions, is considered for dynamic non-nested models. We propose a robust test which generalizes Lien and Vuong's (1987) test with a Heteroscadasticity/Autocorrelation Consistent (HAC) variance estimator. We use the fixed-b asymptotics developed in Kiefer and Vogelsang (2005) to improve the asymptotic approximation to the sampling distribution of the test statistic. The fixed-b approach is compared with a bootstrap method and the standard normal approximation in Monte Carlo simulations. The fixed-b asymptotics and the bootstrap method are found to be markedly superior to the standard normal approximation. An empirical application for foreign exchange rate forecasting models is presented.

Suggested Citation

  • Choi, Hwan-sik & Kiefer, Nicholas M., 2006. "Robust Model Selection in Dynamic Models with an Application to Comparing Predictive Accuracy," Working Papers 06-09, Cornell University, Center for Analytic Economics.
    • Handle: RePEc:ecl:corcae:06-09
    as

    Download full text from publisher

    File URL: https://cae.economics.cornell.edu/06-09.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Davidson, Russell & MacKinnon, James G, 1981. "Several Tests for Model Specification in the Presence of Alternative Hypotheses," Econometrica, Econometric Society, vol. 49(3), pages 781-793, May.
    2. M. H. Pesaran, 1974. "On the General Problem of Model Selection," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 41(2), pages 153-171.
    3. Hansen, Peter Reinhard, 2005. "A Test for Superior Predictive Ability," Journal of Business & Economic Statistics, American Statistical Association, vol. 23, pages 365-380, October.
    4. McAleer, Michael, 1995. "The significance of testing empirical non-nested models," Journal of Econometrics, Elsevier, vol. 67(1), pages 149-171, May.
    5. Hansen, Bruce E., 2005. "Challenges For Econometric Model Selection," Econometric Theory, Cambridge University Press, vol. 21(1), pages 60-68, February.
    6. Lien, Donald & Vuong, Quang H., 1987. "Selecting the best linear regression model : A classical approach," Journal of Econometrics, Elsevier, vol. 35(1), pages 3-23, May.
    7. P. C. B. Phillips & S. N. Durlauf, 1986. "Multiple Time Series Regression with Integrated Processes," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 53(4), pages 473-495.
    8. Nicholas M. Kiefer & Timothy J. Vogelsang & Helle Bunzel, 2000. "Simple Robust Testing of Regression Hypotheses," Econometrica, Econometric Society, vol. 68(3), pages 695-714, May.
    9. Godfrey, L. G. & Pesaran, M. H., 1983. "Tests of non-nested regression models: Small sample adjustments and Monte Carlo evidence," Journal of Econometrics, Elsevier, vol. 21(1), pages 133-154, January.
    10. Kiefer, Nicholas M. & Vogelsang, Timothy J., 2005. "A New Asymptotic Theory For Heteroskedasticity-Autocorrelation Robust Tests," Econometric Theory, Cambridge University Press, vol. 21(6), pages 1130-1164, December.
    11. Yanqin Fan & Qi Li, 1995. "Bootstrapping J-type tests for non-nested regression models," Economics Letters, Elsevier, vol. 48(2), pages 107-112, May.
    12. Kiefer, Nicholas M. & Vogelsang, Timothy J., 2002. "Heteroskedasticity-Autocorrelation Robust Testing Using Bandwidth Equal To Sample Size," Econometric Theory, Cambridge University Press, vol. 18(6), pages 1350-1366, December.
    13. Hall, Peter & Horowitz, Joel L, 1996. "Bootstrap Critical Values for Tests Based on Generalized-Method-of-Moments Estimators," Econometrica, Econometric Society, vol. 64(4), pages 891-916, July.
    14. Godfrey, L. G., 1998. "Tests of non-nested regression models some results on small sample behaviour and the bootstrap," Journal of Econometrics, Elsevier, vol. 84(1), pages 59-74, May.
    15. Michael Jansson, 2004. "The Error in Rejection Probability of Simple Autocorrelation Robust Tests," Econometrica, Econometric Society, vol. 72(3), pages 937-946, May.
    16. Harvey, David & Leybourne, Stephen & Newbold, Paul, 1997. "Testing the equality of prediction mean squared errors," International Journal of Forecasting, Elsevier, vol. 13(2), pages 281-291, June.
    17. Douglas Rivers & Quang Vuong, 2002. "Model selection tests for nonlinear dynamic models," Econometrics Journal, Royal Economic Society, vol. 5(1), pages 1-39, June.
    18. Gourieroux, C. & Monfort, A., 1986. "Testing non-nested hypotheses," Handbook of Econometrics, in: R. F. Engle & D. McFadden (ed.), Handbook of Econometrics, edition 1, volume 4, chapter 44, pages 2583-2637, Elsevier.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. George Batta & Ananda Ganguly & Joshua George Rosett, 2014. "Disclosure-Derived Financial Statement Adjustments in Equity Valuation," Review of Pacific Basin Financial Markets and Policies (RPBFMP), World Scientific Publishing Co. Pte. Ltd., vol. 17(02), pages 1-39.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Davidson, Russell & MacKinnon, James G., 2002. "Bootstrap J tests of nonnested linear regression models," Journal of Econometrics, Elsevier, vol. 109(1), pages 167-193, July.
    2. J. M. C. Santos Silva, 2001. "A score test for non-nested hypotheses with applications to discrete data models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 16(5), pages 577-597.
    3. Godfrey, L. G., 1998. "Tests of non-nested regression models some results on small sample behaviour and the bootstrap," Journal of Econometrics, Elsevier, vol. 84(1), pages 59-74, May.
    4. Debarsy, Nicolas & Ertur, Cem, 2019. "Interaction matrix selection in spatial autoregressive models with an application to growth theory," Regional Science and Urban Economics, Elsevier, vol. 75(C), pages 49-69.
    5. Peter C. B. Phillips & Xiaohu Wang & Yonghui Zhang, 2019. "HAR Testing for Spurious Regression in Trend," Econometrics, MDPI, vol. 7(4), pages 1-28, December.
    6. Sun, Yixiao & Kim, Min Seong, 2012. "Simple and powerful GMM over-identification tests with accurate size," Journal of Econometrics, Elsevier, vol. 166(2), pages 267-281.
    7. Jin, Fei & Lee, Lung-fei, 2013. "Cox-type tests for competing spatial autoregressive models with spatial autoregressive disturbances," Regional Science and Urban Economics, Elsevier, vol. 43(4), pages 590-616.
    8. Sun, Yixiao X & Phillips, Peter C. B. & Jin, Sainan, 2005. "Optimal Bandwidth Selection in Heteroskedasticity-Autocorrelation Robust Testing∗," University of California at San Diego, Economics Working Paper Series qt16b3j2hd, Department of Economics, UC San Diego.
    9. Xiaofeng Shao, 2010. "A self‐normalized approach to confidence interval construction in time series," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 72(3), pages 343-366, June.
    10. West, Kenneth D., 2001. "Encompassing tests when no model is encompassing," Journal of Econometrics, Elsevier, vol. 105(1), pages 287-308, November.
    11. Bernard Fingleton & Silvia Palombi, 2016. "Bootstrap J -Test for Panel Data Models with Spatially Dependent Error Components, a Spatial Lag and Additional Endogenous Variables," Spatial Economic Analysis, Taylor & Francis Journals, vol. 11(1), pages 7-26, March.
    12. Hagemann, Andreas, 2012. "A simple test for regression specification with non-nested alternatives," Journal of Econometrics, Elsevier, vol. 166(2), pages 247-254.
    13. Kim, Min Seong & Sun, Yixiao & Yang, Jingjing, 2017. "A fixed-bandwidth view of the pre-asymptotic inference for kernel smoothing with time series data," Journal of Econometrics, Elsevier, vol. 197(2), pages 298-322.
    14. Sun, Yixiao, 2014. "Let’s fix it: Fixed-b asymptotics versus small-b asymptotics in heteroskedasticity and autocorrelation robust inference," Journal of Econometrics, Elsevier, vol. 178(P3), pages 659-677.
    15. Casini, Alessandro, 2023. "Theory of evolutionary spectra for heteroskedasticity and autocorrelation robust inference in possibly misspecified and nonstationary models," Journal of Econometrics, Elsevier, vol. 235(2), pages 372-392.
    16. Francisco Cribari-Neto & Sadraque E.F. Lucena, 2015. "Nonnested hypothesis testing in the class of varying dispersion beta regressions," Journal of Applied Statistics, Taylor & Francis Journals, vol. 42(5), pages 967-985, May.
    17. Kim, Min Seong & Sun, Yixiao, 2013. "Heteroskedasticity and spatiotemporal dependence robust inference for linear panel models with fixed effects," Journal of Econometrics, Elsevier, vol. 177(1), pages 85-108.
    18. Han, Xiaoyi & Lee, Lung-fei, 2013. "Model selection using J-test for the spatial autoregressive model vs. the matrix exponential spatial model," Regional Science and Urban Economics, Elsevier, vol. 43(2), pages 250-271.
    19. Preinerstorfer, David & Pötscher, Benedikt M., 2016. "On Size And Power Of Heteroskedasticity And Autocorrelation Robust Tests," Econometric Theory, Cambridge University Press, vol. 32(2), pages 261-358, April.
    20. Cheol-Keun Cho & Timothy J. Vogelsang, 2016. "Fixed- b Inference for Testing Structural Change in a Time Series Regression," Econometrics, MDPI, vol. 5(1), pages 1-26, December.

    More about this item

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General
    • C15 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Statistical Simulation Methods: General
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ecl:corcae:06-09. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/cacorus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.