IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v54y2010i12p3194-3211.html
   My bibliography  Save this article

Model selection strategies for identifying most relevant covariates in homoscedastic linear models

Author

Listed:
  • Min, Aleksey
  • Holzmann, Hajo
  • Czado, Claudia

Abstract

A new method in two variations for the identification of most relevant covariates in linear models with homoscedastic errors is proposed. In contrast to many known selection criteria, the method is based on an interpretable scaled quantity. This quantity measures a maximal relative error one makes by selecting covariates from a given set of all available covariates. The proposed model selection procedures rely on asymptotic normality of test statistics, and therefore normality of the errors in the regression model is not required. In a simulation study the performance of the suggested methods along with the performance of the standard model selection criteria AIC, BIC, Lasso and relaxed Lasso is examined. The simulation study illustrates the favorable performance of the proposed method as compared to the above reference criteria, especially when regression effects possess influence of several orders in magnitude. The accuracy of the normal approximation to the test statistics is also investigated; it has been already satisfactory for sample sizes 50 and 100. As an illustration the US college spending data from 1994 is analyzed.

Suggested Citation

  • Min, Aleksey & Holzmann, Hajo & Czado, Claudia, 2010. "Model selection strategies for identifying most relevant covariates in homoscedastic linear models," Computational Statistics & Data Analysis, Elsevier, vol. 54(12), pages 3194-3211, December.
    • Handle: RePEc:eee:csdana:v:54:y:2010:i:12:p:3194-3211
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167-9473(09)00322-3
    Download Restriction: Full text for ScienceDirect subscribers only.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Meinshausen, Nicolai, 2007. "Relaxed Lasso," Computational Statistics & Data Analysis, Elsevier, vol. 52(1), pages 374-393, September.
    2. Yancey, T A & Judge, G G & Bock, M E, 1973. "Wallace's Weak Mean Square Error Criterion for Testing Linear Restrictions in Regression: A Tighter Bound," Econometrica, Econometric Society, vol. 41(6), pages 1203-1206, November.
    3. David J. Spiegelhalter & Nicola G. Best & Bradley P. Carlin & Angelika Van Der Linde, 2002. "Bayesian measures of model complexity and fit," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 64(4), pages 583-639, October.
    4. Wang, Hansheng & Leng, Chenlei, 2008. "A note on adaptive group lasso," Computational Statistics & Data Analysis, Elsevier, vol. 52(12), pages 5277-5286, August.
    5. Holger Dette & Axel Munk, 2003. "Some Methodological Aspects of Validation of Models in Nonparametric Regression," Statistica Neerlandica, Netherlands Society for Statistics and Operations Research, vol. 57(2), pages 207-244, May.
    6. Wallace, T D, 1972. "Weaker Criteria and Tests for Linear Restrictions in Regression," Econometrica, Econometric Society, vol. 40(4), pages 689-698, July.
    7. Leeb, Hannes & Pötscher, Benedikt M., 2003. "The Finite-Sample Distribution Of Post-Model-Selection Estimators And Uniform Versus Nonuniform Approximations," Econometric Theory, Cambridge University Press, vol. 19(1), pages 100-142, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ruth M. Pfeiffer & Andrew Redd & Raymond J. Carroll, 2017. "On the impact of model selection on predictor identification and parameter inference," Computational Statistics, Springer, vol. 32(2), pages 667-690, June.
    2. Tutz, Gerhard & Pößnecker, Wolfgang & Uhlmann, Lorenz, 2015. "Variable selection in general multinomial logit models," Computational Statistics & Data Analysis, Elsevier, vol. 82(C), pages 207-222.
    3. Buddhavarapu, Prasad & Bansal, Prateek & Prozzi, Jorge A., 2021. "A new spatial count data model with time-varying parameters," Transportation Research Part B: Methodological, Elsevier, vol. 150(C), pages 566-586.
    4. Mumtaz, Haroon & Theodoridis, Konstantinos, 2017. "Common and country specific economic uncertainty," Journal of International Economics, Elsevier, vol. 105(C), pages 205-216.
    5. Christina Leuker & Thorsten Pachur & Ralph Hertwig & Timothy J. Pleskac, 2019. "Do people exploit risk–reward structures to simplify information processing in risky choice?," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 5(1), pages 76-94, August.
    6. Rubio, F.J. & Steel, M.F.J., 2011. "Inference for grouped data with a truncated skew-Laplace distribution," Computational Statistics & Data Analysis, Elsevier, vol. 55(12), pages 3218-3231, December.
    7. Mkhadri, Abdallah & Ouhourane, Mohamed, 2013. "An extended variable inclusion and shrinkage algorithm for correlated variables," Computational Statistics & Data Analysis, Elsevier, vol. 57(1), pages 631-644.
    8. Alessandri, Piergiorgio & Mumtaz, Haroon, 2019. "Financial regimes and uncertainty shocks," Journal of Monetary Economics, Elsevier, vol. 101(C), pages 31-46.
    9. Svetlana V. Tishkovskaya & Paul G. Blackwell, 2021. "Bayesian estimation of heterogeneous environments from animal movement data," Environmetrics, John Wiley & Sons, Ltd., vol. 32(6), September.
    10. Leonardo Oliveira Martins & Hirohisa Kishino, 2010. "Distribution of distances between topologies and its effect on detection of phylogenetic recombination," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 62(1), pages 145-159, February.
    11. Tamal Ghosh & Malay Ghosh & Jerry J. Maples & Xueying Tang, 2022. "Multivariate Global-Local Priors for Small Area Estimation," Stats, MDPI, vol. 5(3), pages 1-16, July.
    12. Wan, Alan T.K. & Zhang, Xinyu & Zou, Guohua, 2010. "Least squares model averaging by Mallows criterion," Journal of Econometrics, Elsevier, vol. 156(2), pages 277-283, June.
    13. Eibich, Peter & Ziebarth, Nicolas, 2014. "Examining the Structure of Spatial Health Effects in Germany Using Hierarchical Bayes Models," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 49, pages 305-320.
    14. Wu, Ji & Guo, Mengmeng & Chen, Minghua & Jeon, Bang Nam, 2019. "Market power and risk-taking of banks: Some semiparametric evidence from emerging economies," Emerging Markets Review, Elsevier, vol. 41(C).
    15. repec:jss:jstsof:21:i08 is not listed on IDEAS
    16. Deng, Yaguo & Lopes Moreira Da Veiga, María Helena & Wiper, Michael Peter, 2016. "Efficiency evaluation of Spanish hotel chains," DES - Working Papers. Statistics and Econometrics. WS 23897, Universidad Carlos III de Madrid. Departamento de Estadística.
    17. Castle Jennifer L. & Doornik Jurgen A & Hendry David F., 2011. "Evaluating Automatic Model Selection," Journal of Time Series Econometrics, De Gruyter, vol. 3(1), pages 1-33, February.
    18. Cathy W. S. Chen & Sangyeol Lee, 2017. "Bayesian causality test for integer-valued time series models with applications to climate and crime data," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 66(4), pages 797-814, August.
    19. Makoto Chikaraishi & Akimasa Fujiwara & Junyi Zhang & Kay Axhausen, 2011. "Identifying variations and co-variations in discrete choice models," Transportation, Springer, vol. 38(6), pages 993-1016, November.
    20. Galatia Cleanthous & Emilio Porcu & Philip White, 2021. "Regularity and approximation of Gaussian random fields evolving temporally over compact two-point homogeneous spaces," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 30(4), pages 836-860, December.
    21. Dong, C. & Li, S., 2021. "Specification Lasso and an Application in Financial Markets," Cambridge Working Papers in Economics 2139, Faculty of Economics, University of Cambridge.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:54:y:2010:i:12:p:3194-3211. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.