IDEAS home Printed from https://ideas.repec.org/a/kap/compec/v54y2019i1d10.1007_s10614-017-9682-8.html
   My bibliography  Save this article

Performances of Model Selection Criteria When Variables are Ill Conditioned

Author

Listed:
  • Peter S. Karlsson

    (Linnaeus University)

  • Lars Behrenz

    (Linnaeus University)

  • Ghazi Shukur

    (Linnaeus University
    Jönköping University)

Abstract

Model selection criteria are often used to find a “proper” model for the data under investigation when building models in cases in which the dependent or explained variables are assumed to be functions of several independent or explanatory variables. For this purpose, researchers have suggested using a large number of such criteria. These criteria have been shown to act differently, under the same or different conditions, when trying to select the “correct” number of explanatory variables to be included in a given model; this, unfortunately, leads to severe problems and confusion for researchers. In this paper, using Monte Carlo methods, we investigate the properties of four of the most common criteria under a number of realistic situations. These criteria are the adjusted coefficient of determination ( $$\hbox {R}^{2}$$ R 2 -adj), Akaike’s information criterion (AIC), the Hannan–Quinn information criterion (HQC) and the Bayesian information criterion (BIC). The results from this investigation indicate that the HQC outperforms the BIC, the AIC and the $$\hbox {R}^{2}$$ R 2 -adj under specific circumstances. None of them perform satisfactorily, however, when the degree of multicollinearity is high, the sample sizes are small or when the fit of the model is poor (i.e., there is a low $$\hbox {R}^{2})$$ R 2 ) . In the presence of all these factors, the criteria perform very badly and are not very useful. In these cases, the criteria are often not able to select the true model.

Suggested Citation

  • Peter S. Karlsson & Lars Behrenz & Ghazi Shukur, 2019. "Performances of Model Selection Criteria When Variables are Ill Conditioned," Computational Economics, Springer;Society for Computational Economics, vol. 54(1), pages 77-98, June.
  • Handle: RePEc:kap:compec:v:54:y:2019:i:1:d:10.1007_s10614-017-9682-8
    DOI: 10.1007/s10614-017-9682-8
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10614-017-9682-8
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s10614-017-9682-8?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Hamparsum Bozdogan, 1987. "Model selection and Akaike's Information Criterion (AIC): The general theory and its analytical extensions," Psychometrika, Springer;The Psychometric Society, vol. 52(3), pages 345-370, September.
    2. Edgerton, David L., 1996. "Should stochastic or non-stochastic exogenous variables be used in Monte Carlo experiments?," Economics Letters, Elsevier, vol. 53(2), pages 153-159, November.
    3. Atkinson, A. C., 1981. "Likelihood ratios, posterior odds and information criteria," Journal of Econometrics, Elsevier, vol. 16(1), pages 15-20, May.
    4. Amemiya, Takeshi, 1980. "Selection of Regressors," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 21(2), pages 331-354, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Luc Anselin, 1988. "Model Validation in Spatial Econometrics: A Review and Evaluation of Alternative Approaches," International Regional Science Review, , vol. 11(3), pages 279-316, December.
    2. Todd Guilfoos & Priya Thomas & Sonja Kolstoe, 2024. "Estimating habit‐forming and variety‐seeking behavior: Valuation of recreational birdwatching," American Journal of Agricultural Economics, John Wiley & Sons, vol. 106(3), pages 1193-1216, May.
    3. Ioana Gutu & Daniela Tatiana Agheorghiesei & Alexandru Tugui, 2023. "Assessment of a Workforce Sustainability Tool through Leadership and Digitalization," IJERPH, MDPI, vol. 20(2), pages 1-30, January.
    4. Adrian Bruhin & Ernst Fehr & Daniel Schunk, 2019. "The many Faces of Human Sociality: Uncovering the Distribution and Stability of Social Preferences," Journal of the European Economic Association, European Economic Association, vol. 17(4), pages 1025-1069.
    5. Daniela Andreini & Diego Rinallo & Giuseppe Pedeliento & Mara Bergamaschi, 2017. "Brands and Religion in the Secularized Marketplace and Workplace: Insights from the Case of an Italian Hospital Renamed After a Roman Catholic Pope," Journal of Business Ethics, Springer, vol. 141(3), pages 529-550, March.
    6. David Genesove & Christopher Mayer, 2001. "Loss Aversion and Seller Behavior: Evidence from the Housing Market," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 116(4), pages 1233-1260.
    7. Fernandez, Carmen & Ley, Eduardo & Steel, Mark F. J., 2001. "Benchmark priors for Bayesian model averaging," Journal of Econometrics, Elsevier, vol. 100(2), pages 381-427, February.
    8. S. A. Abu Bakar & Saralees Nadarajah & Z. A. Absl Kamarul Adzhar, 2018. "Loss modeling using Burr mixtures," Empirical Economics, Springer, vol. 54(4), pages 1503-1516, June.
    9. Byrd, T. A. & Marshall, T. E., 1997. "Relating information technology investment to organizational performance: a causal model analysis," Omega, Elsevier, vol. 25(1), pages 43-56, February.
    10. Castle Jennifer L. & Doornik Jurgen A & Hendry David F., 2011. "Evaluating Automatic Model Selection," Journal of Time Series Econometrics, De Gruyter, vol. 3(1), pages 1-33, February.
    11. David F. Hendry & Hans-Martin Krolzig, 2005. "The Properties of Automatic "GETS" Modelling," Economic Journal, Royal Economic Society, vol. 115(502), pages C32-C61, 03.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:kap:compec:v:54:y:2019:i:1:d:10.1007_s10614-017-9682-8. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.