IDEAS home Printed from https://ideas.repec.org/p/msh/ebswps/2011-7.html
   My bibliography  Save this paper

A New Procedure For Multiple Testing Of Econometric Models

Author

Listed:
  • Maxwell L. King
  • Xibin Zhang
  • Muhammad Akram

Abstract

A significant role for hypothesis testing in econometrics involves diagnostic checking. When checking the adequacy of a chosen model, researchers typically employ a range of diagnostic tests, each of which is designed to detect a particular form of model inadequacy. A major problem is how best to control the overall probability of rejecting the model when it is true and multiple test statistics are used. This paper presents a new multiple testing procedure, which involves checking whether the calculated values of the diagnostic statistics are consistent with the postulated model being true. This is done through a combination of bootstrapping to obtain a multivariate kernel density estimator of the joint density of the test statistics under the null hypothesis and Monte Carlo simulations to obtain a p value using this kernel density. We prove that under some regularity conditions, the estimated p value of our test procedure is a consistent estimate of the true p value. The proposed testing procedure is applied to tests for autocorrelation in an observed time series, for normality, and for model misspecification through the information matrix. We find that our testing procedure has correct or nearly correct sizes and good powers, particular for more complicated testing problems. We believe it is the first good method for calculating the overall p value for a vector of test statistics based on simulation.

Suggested Citation

  • Maxwell L. King & Xibin Zhang & Muhammad Akram, 2011. "A New Procedure For Multiple Testing Of Econometric Models," Monash Econometrics and Business Statistics Working Papers 7/11, Monash University, Department of Econometrics and Business Statistics.
    • Handle: RePEc:msh:ebswps:2011-7
    as

    Download full text from publisher

    File URL: http://business.monash.edu/econometrics-and-business-statistics/research/publications/ebs/wp7-11.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Zhang, Xibin & King, Maxwell L. & Hyndman, Rob J., 2006. "A Bayesian approach to bandwidth selection for multivariate kernel density estimation," Computational Statistics & Data Analysis, Elsevier, vol. 50(11), pages 3009-3031, July.
    2. White, Halbert, 1982. "Maximum Likelihood Estimation of Misspecified Models," Econometrica, Econometric Society, vol. 50(1), pages 1-25, January.
    3. Lancaster, Tony, 1984. "The Covariance Matrix of the Information Matrix Test," Econometrica, Econometric Society, vol. 52(4), pages 1051-1053, July.
    4. Horowitz, J. L., 1995. "Bootstrap Methods In Econometrics: Theory And Numerical Performance," SFB 373 Discussion Papers 1995,63, Humboldt University of Berlin, Interdisciplinary Research Project 373: Quantification and Simulation of Economic Processes.
    5. Alastair Hall, 1987. "The Information Matrix Test for the Linear Model," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 54(2), pages 257-263.
    6. Horowitz, Joel L., 1994. "Bootstrap-based critical values for the information matrix test," Journal of Econometrics, Elsevier, vol. 61(2), pages 395-411, April.
    7. Davidson, Russell & MacKinnon, James G, 1992. "A New Form of the Information Matrix Test," Econometrica, Econometric Society, vol. 60(1), pages 145-157, January.
    8. Unknown, 1986. "Letters," Choices: The Magazine of Food, Farm, and Resource Issues, Agricultural and Applied Economics Association, vol. 1(4), pages 1-9.
    9. Joel L. Horowitz, 1996. "Bootstrap Methods in Econometrics: Theory and Numerical Performance," Econometrics 9602009, University Library of Munich, Germany, revised 05 Mar 1996.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jin Seo Cho & Halbert White, 2014. "Testing the Equality of Two Positive-Definite Matrices with Application to Information Matrix Testing," Working papers 2014rwp-67, Yonsei University, Yonsei Economics Research Institute.
    2. Julia Polak & Maxwell L. King & Xibin Zhang, 2014. "A Model Validation Procedure," Monash Econometrics and Business Statistics Working Papers 21/14, Monash University, Department of Econometrics and Business Statistics.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. King, Maxwell L. & Zhang, Xibin & Akram, Muhammad, 2020. "Hypothesis testing based on a vector of statistics," Journal of Econometrics, Elsevier, vol. 219(2), pages 425-455.
    2. Davidson, Russell & MacKinnon, James G, 1998. "Graphical Methods for Investigating the Size and Power of Hypothesis Tests," The Manchester School of Economic & Social Studies, University of Manchester, vol. 66(1), pages 1-26, January.
    3. Riccardo Lucchetti & Claudia Pigini, 2013. "A test for bivariate normality with applications in microeconometric models," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 22(4), pages 535-572, November.
    4. Dhaene, Geert & Hoorelbeke, Dirk, 2004. "The information matrix test with bootstrap-based covariance matrix estimation," Economics Letters, Elsevier, vol. 82(3), pages 341-347, March.
    5. Wanling Huang & Artem Prokhorov, 2014. "A Goodness-of-fit Test for Copulas," Econometric Reviews, Taylor & Francis Journals, vol. 33(7), pages 751-771, October.
    6. Stomberg, Christopher & White, Halbert, 2000. "Bootstrapping the Information Matrix Test," University of California at San Diego, Economics Working Paper Series qt158451cr, Department of Economics, UC San Diego.
    7. Choi, Hwan-sik, 2016. "Information theory for maximum likelihood estimation of diffusion models," Journal of Econometrics, Elsevier, vol. 191(1), pages 110-128.
    8. Li, Yong & Yu, Jun & Zeng, Tao, 2018. "Specification tests based on MCMC output," Journal of Econometrics, Elsevier, vol. 207(1), pages 237-260.
    9. Dirk Hoorelbeke, 2004. "Bootstrap correcting the score test," Econometric Society 2004 North American Summer Meetings 228, Econometric Society.
    10. repec:ebl:ecbull:v:3:y:2008:i:5:p:1-7 is not listed on IDEAS
    11. Joachim Zietz, 2006. "Detecting neglected parameter heterogeneity with Chow tests," Applied Economics Letters, Taylor & Francis Journals, vol. 13(6), pages 369-374.
    12. Chesher, Andrew & Dhaene, Geert & Gouriéroux, Christian & Scaillet, Olivier, 1999. "Bartlett Identities Tests," LIDAM Discussion Papers IRES 1999019, Université catholique de Louvain, Institut de Recherches Economiques et Sociales (IRES).
    13. James G. MacKinnon & Russell Davidson, 1996. "The Size And Power Of Bootstrap Tests," Working Paper 932, Economics Department, Queen's University.
    14. Boldea, Otilia & Magnus, Jan R., 2009. "Maximum Likelihood Estimation of the Multivariate Normal Mixture Model," Journal of the American Statistical Association, American Statistical Association, vol. 104(488), pages 1539-1549.
    15. K. Chua & S. Ong, 2013. "Test of misspecification with application to negative binomial distribution," Computational Statistics, Springer, vol. 28(3), pages 993-1009, June.
    16. Daisuke Nagakura, 2008. "A note on the relationship between the information matrx test and a score test for parameter constancy," Economics Bulletin, AccessEcon, vol. 3(5), pages 1-7.
    17. Christian de Peretti, 2003. "Bilateral Bootstrap Tests for Long Memory: An Application to the Silver Market," Computational Economics, Springer;Society for Computational Economics, vol. 22(2), pages 187-212, October.
    18. Cho, Jin Seo & White, Halbert, 2010. "Testing for unobserved heterogeneity in exponential and Weibull duration models," Journal of Econometrics, Elsevier, vol. 157(2), pages 458-480, August.
    19. Alessandra Canepa & Raymond O'Brien, 2000. "The Size and Power of Bootstrap Tests for Linear Restrictions in Misspecified Cointegrating Relationships," Econometric Society World Congress 2000 Contributed Papers 1807, Econometric Society.
    20. Cho, Jin Seo & Phillips, Peter C.B., 2018. "Pythagorean generalization of testing the equality of two symmetric positive definite matrices," Journal of Econometrics, Elsevier, vol. 202(1), pages 45-56.
    21. MacKinnon, James G, 1992. "Model Specification Tests and Artificial Regressions," Journal of Economic Literature, American Economic Association, vol. 30(1), pages 102-146, March.

    More about this item

    Keywords

    Bootstrapping; consistency; information matrix test; Markov chain Monte Carlo simulation; multivariate kernel density; normality; serial correlation; test vector;
    All these keywords.

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:msh:ebswps:2011-7. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Professor Xibin Zhang (email available below). General contact details of provider: https://edirc.repec.org/data/dxmonau.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.