IDEAS home Printed from https://ideas.repec.org/p/pra/mprapa/41459.html
   My bibliography  Save this paper

Testing in the Presence of Nuisance Parameters: Some Comments on Tests Post-Model-Selection and Random Critical Values

Author

Listed:
  • Leeb, Hannes
  • Pötscher, Benedikt M.

Abstract

We point out that the ideas underlying some test procedures recently proposed for testing post-model-selection (and for some other test problems) in the econometrics literature have been around for quite some time in the statistics literature. We also sharpen some of these results in the statistics literature and show that some of the proposals in the econometrics literature lead to tests that do not have the claimed size properties.

Suggested Citation

  • Leeb, Hannes & Pötscher, Benedikt M., 2012. "Testing in the Presence of Nuisance Parameters: Some Comments on Tests Post-Model-Selection and Random Critical Values," MPRA Paper 41459, University Library of Munich, Germany.
    • Handle: RePEc:pra:mprapa:41459
    as

    Download full text from publisher

    File URL: https://mpra.ub.uni-muenchen.de/41459/1/MPRA_paper_41459.pdf
    File Function: original version
    Download Restriction: no
    File URL: https://mpra.ub.uni-muenchen.de/47102/1/MPRA_paper_47102.pdf
    File Function: revised version
    Download Restriction: no
    File URL: https://mpra.ub.uni-muenchen.de/56276/9/MPRA_paper_56276.pdf
    File Function: revised version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. DiTraglia, Francis J., 2016. "Using invalid instruments on purpose: Focused moment selection and averaging for GMM," Journal of Econometrics, Elsevier, vol. 195(2), pages 187-208.
    2. Kabaila, Paul & Leeb, Hannes, 2006. "On the Large-Sample Minimal Coverage Probability of Confidence Intervals After Model Selection," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 619-629, June.
    3. Leeb, Hannes & Pötscher, Benedikt M., 2003. "The Finite-Sample Distribution Of Post-Model-Selection Estimators And Uniform Versus Nonuniform Approximations," Econometric Theory, Cambridge University Press, vol. 19(1), pages 100-142, February.
    4. Leeb, Hannes & Pötscher, Benedikt M., 2005. "Model Selection And Inference: Facts And Fiction," Econometric Theory, Cambridge University Press, vol. 21(1), pages 21-59, February.
    5. Donald W. K. Andrews & Patrik Guggenberger, 2009. "Hybrid and Size-Corrected Subsampling Methods," Econometrica, Econometric Society, vol. 77(3), pages 721-762, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Bachoc, Francois & Leeb, Hannes & Pötscher, Benedikt M., 2014. "Valid confidence intervals for post-model-selection predictors," MPRA Paper 60643, University Library of Munich, Germany.
    2. DiTraglia, Francis J., 2016. "Using invalid instruments on purpose: Focused moment selection and averaging for GMM," Journal of Econometrics, Elsevier, vol. 195(2), pages 187-208.
    3. McCloskey, Adam, 2017. "Bonferroni-based size-correction for nonstandard testing problems," Journal of Econometrics, Elsevier, vol. 200(1), pages 17-35.
    4. Liu, Chu-An, 2015. "Distribution theory of the least squares averaging estimator," Journal of Econometrics, Elsevier, vol. 186(1), pages 142-159.
    5. Francis J. DiTraglia, 2011. "Using Invalid Instruments on Purpose: Focused Moment Selection and Averaging for GMM, Second Version," PIER Working Paper Archive 14-045, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania, revised 09 Dec 2014.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Leeb, Hannes & Pötscher, Benedikt M. & Ewald, Karl, 2014. "On various confidence intervals post-model-selection," MPRA Paper 52858, University Library of Munich, Germany.
    2. Liu, Chu-An, 2015. "Distribution theory of the least squares averaging estimator," Journal of Econometrics, Elsevier, vol. 186(1), pages 142-159.
    3. Doko Tchatoka, Firmin & Wang, Wenjie, 2021. "Uniform Inference after Pretesting for Exogeneity with Heteroskedastic Data," MPRA Paper 106408, University Library of Munich, Germany.
    4. Liu, Chu-An, 2012. "A plug-in averaging estimator for regressions with heteroskedastic errors," MPRA Paper 41414, University Library of Munich, Germany.
    5. Cheng, Xu & Liao, Zhipeng, 2015. "Select the valid and relevant moments: An information-based LASSO for GMM with many moments," Journal of Econometrics, Elsevier, vol. 186(2), pages 443-464.
    6. Ruth M. Pfeiffer & Andrew Redd & Raymond J. Carroll, 2017. "On the impact of model selection on predictor identification and parameter inference," Computational Statistics, Springer, vol. 32(2), pages 667-690, June.
    7. Firmin Doko Tchatoka & Wenjie Wang, 2020. "Uniform Inference after Pretesting for Exogeneity," School of Economics and Public Policy Working Papers 2020-05, University of Adelaide, School of Economics and Public Policy.
    8. Wan, Alan T.K. & Zhang, Xinyu & Zou, Guohua, 2010. "Least squares model averaging by Mallows criterion," Journal of Econometrics, Elsevier, vol. 156(2), pages 277-283, June.
    9. Castle Jennifer L. & Doornik Jurgen A & Hendry David F., 2011. "Evaluating Automatic Model Selection," Journal of Time Series Econometrics, De Gruyter, vol. 3(1), pages 1-33, February.
    10. Timothy B. Armstrong & Michal Kolesár, 2021. "Sensitivity analysis using approximate moment condition models," Quantitative Economics, Econometric Society, vol. 12(1), pages 77-108, January.
    11. Shaobo Jin, 2022. "Frequentist Model Averaging in Structure Equation Model With Ordinal Data," Psychometrika, Springer;The Psychometric Society, vol. 87(3), pages 1130-1145, September.
    12. Hassler, Uwe, 2010. "Testing regression coefficients after model selection through sign restrictions," Economics Letters, Elsevier, vol. 107(2), pages 220-223, May.
    13. Andrews, Donald W.K. & Guggenberger, Patrik, 2009. "Incorrect asymptotic size of subsampling procedures based on post-consistent model selection estimators," Journal of Econometrics, Elsevier, vol. 152(1), pages 19-27, September.
    14. Zhang, Xinyu & Yu, Jihai, 2018. "Spatial weights matrix selection and model averaging for spatial autoregressive models," Journal of Econometrics, Elsevier, vol. 203(1), pages 1-18.
    15. Xu Cheng & Zhipeng Liao, 2012. "Select the Valid and Relevant Moments: A One-Step Procedure for GMM with Many Moments," PIER Working Paper Archive 12-045, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania.
    16. Matei Demetrescu & Uwe Hassler & Vladimir Kuzin, 2011. "Pitfalls of post-model-selection testing: experimental quantification," Empirical Economics, Springer, vol. 40(2), pages 359-372, April.
    17. Karthik Muralidharan & Mauricio Romero & Kaspar Wüthrich, 2019. "Factorial Designs, Model Selection, and (Incorrect) Inference in Randomized Experiments," NBER Working Papers 26562, National Bureau of Economic Research, Inc.
    18. Andrews, Donald W.K. & Cheng, Xu & Guggenberger, Patrik, 2020. "Generic results for establishing the asymptotic size of confidence sets and tests," Journal of Econometrics, Elsevier, vol. 218(2), pages 496-531.
    19. Castle, Jennifer L. & Doornik, Jurgen A. & Hendry, David F., 2012. "Model selection when there are multiple breaks," Journal of Econometrics, Elsevier, vol. 169(2), pages 239-246.
    20. Bachoc, Francois & Leeb, Hannes & Pötscher, Benedikt M., 2014. "Valid confidence intervals for post-model-selection predictors," MPRA Paper 60643, University Library of Munich, Germany.

    More about this item

    Keywords

    Tests in presence of nuisance parameters; inference post model selection;

    JEL classification:

    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C01 - Mathematical and Quantitative Methods - - General - - - Econometrics

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pra:mprapa:41459. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Joachim Winter (email available below). General contact details of provider: https://edirc.repec.org/data/vfmunde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.