IDEAS home Printed from https://ideas.repec.org/a/bpj/ijbist/v6y2010i1n16.html
   My bibliography  Save this article

Estimating Multilevel Logistic Regression Models When the Number of Clusters is Low: A Comparison of Different Statistical Software Procedures

Author

Listed:
  • Austin Peter C

    (Institute for Clinical Evaluative Sciences)

Abstract

Multilevel logistic regression models are increasingly being used to analyze clustered data in medical, public health, epidemiological, and educational research. Procedures for estimating the parameters of such models are available in many statistical software packages. There is currently little evidence on the minimum number of clusters necessary to reliably fit multilevel regression models. We conducted a Monte Carlo study to compare the performance of different statistical software procedures for estimating multilevel logistic regression models when the number of clusters was low. We examined procedures available in BUGS, HLM, R, SAS, and Stata. We found that there were qualitative differences in the performance of different software procedures for estimating multilevel logistic models when the number of clusters was low. Among the likelihood-based procedures, estimation methods based on adaptive Gauss-Hermite approximations to the likelihood (glmer in R and xtlogit in Stata) or adaptive Gaussian quadrature (Proc NLMIXED in SAS) tended to have superior performance for estimating variance components when the number of clusters was small, compared to software procedures based on penalized quasi-likelihood. However, only Bayesian estimation with BUGS allowed for accurate estimation of variance components when there were fewer than 10 clusters. For all statistical software procedures, estimation of variance components tended to be poor when there were only five subjects per cluster, regardless of the number of clusters.

Suggested Citation

  • Austin Peter C, 2010. "Estimating Multilevel Logistic Regression Models When the Number of Clusters is Low: A Comparison of Different Statistical Software Procedures," The International Journal of Biostatistics, De Gruyter, vol. 6(1), pages 1-20, April.
    • Handle: RePEc:bpj:ijbist:v:6:y:2010:i:1:n:16
    DOI: 10.2202/1557-4679.1195
    as

    Download full text from publisher

    File URL: https://doi.org/10.2202/1557-4679.1195
    Download Restriction: For access to full text, subscription to the journal or payment for the individual article is required.
    File URL: https://libkey.io/10.2202/1557-4679.1195?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Germáan Rodríguez & Noreen Goldman, 1995. "An Assessment of Estimation Procedures for Multilevel Models with Binary Responses," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 158(1), pages 73-89, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Erwin Stolz & Hannes Mayerl & Éva Rásky & Wolfgang Freidl, 2019. "Individual and country-level determinants of nursing home admission in the last year of life in Europe," PLOS ONE, Public Library of Science, vol. 14(3), pages 1-10, March.
    2. Kamphuis, Pascal & Glebbeek, Arie C., 2020. "Job (in)security and workers' training decisions: A framing approach," International Journal for Research in Vocational Education and Training (IJRVET), European Research Network in Vocational Education and Training (VETNET), European Educational Research Association, vol. 7(3), pages 361-387.
    3. Diaz, Mireya, 2015. "Performance measures of the bivariate random effects model for meta-analyses of diagnostic accuracy," Computational Statistics & Data Analysis, Elsevier, vol. 83(C), pages 82-90.
    4. Giuseppe Carrà & Cristina Crocamo & Paul Bebbington, 2017. "Gambling, geographical variations and deprivation: findings from the adult psychiatric morbidity survey," International Gambling Studies, Taylor & Francis Journals, vol. 17(3), pages 459-470, September.
    5. Douglas Eadie & Martine Stead & Anne Marie MacKintosh & Susan Murray & Catherine Best & Jamie Pearce & Catherine Tisch & Winfried van der Sluijs & Amanda Amos & Andy MacGregor & Sally Haw, 2016. "Are Retail Outlets Complying with National Legislation to Protect Children from Exposure to Tobacco Displays at Point of Sale? Results from the First Compliance Study in the UK," PLOS ONE, Public Library of Science, vol. 11(3), pages 1-11, March.
    6. Steffen Andreas Schüle & Gabriele Bolte, 2015. "Interactive and Independent Associations between the Socioeconomic and Objective Built Environment on the Neighbourhood Level and Individual Health: A Systematic Review of Multilevel Studies," PLOS ONE, Public Library of Science, vol. 10(4), pages 1-31, April.
    7. L. Bryan, Mark & P. Jenkins, Stephen, 2013. "Regression analysis of country effects using multilevel data: a cautionary tale," ISER Working Paper Series 2013-14, Institute for Social and Economic Research.
    8. Kim, Tae Jun & Vonneilich, Nico & Lüdecke, Daniel & von dem Knesebeck, Olaf, 2017. "Income, financial barriers to health care and public health expenditure: A multilevel analysis of 28 countries," Social Science & Medicine, Elsevier, vol. 176(C), pages 158-165.
    9. Song, Fei (Sophie) & Montabon, Frank & Xu, Yuhang, 2018. "The impact of national culture on corporate adoption of environmental management practices and their effectiveness," International Journal of Production Economics, Elsevier, vol. 205(C), pages 313-328.
    10. Giorgio Di Gessa & Karen Glaser & Debora Price & Eloi Ribe & Anthea Tinker, 2016. "What Drives National Differences in Intensive Grandparental Childcare in Europe?," The Journals of Gerontology: Series B, The Gerontological Society of America, vol. 71(1), pages 141-153.
    11. Pau Baizan & Bruno Arpino & Carlos Eric Delclòs, 2016. "The Effect of Gender Policies on Fertility: The Moderating Role of Education and Normative Context," European Journal of Population, Springer;European Association for Population Studies, vol. 32(1), pages 1-30, February.
    12. Finger, Claudia, 2022. "(Mis)Matched College Aspirations and Expectations: The Role of Social Background and Admission Barriers," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 38(3), pages 472-492.
    13. Nathalie Greenan & Ekaterina Kalugina & Mouhamadou Moustapha Niang, 2017. "Work Organisation and Workforce Vunerability to Non-Employment: Evidence from OECD’s Survey on Adult Skills (PIAAC) [Organisation du travail et vulnérabilité au non-emploi : une étude empirique à p," Working Papers hal-02162457, HAL.
    14. Philipp Poppitz, 2016. "Does self-perceptions and income inequality match?," IMK Working Paper 173-2016, IMK at the Hans Boeckler Foundation, Macroeconomic Policy Institute.
    15. D. Betsy McCoach & Graham G. Rifenbark & Sarah D. Newton & Xiaoran Li & Janice Kooken & Dani Yomtov & Anthony J. Gambino & Aarti Bellara, 2018. "Does the Package Matter? A Comparison of Five Common Multilevel Modeling Software Packages," Journal of Educational and Behavioral Statistics, , vol. 43(5), pages 594-627, October.
    16. Paul Emile Tchinda & Seung-Nam Kim, 2020. "The Paradox of “Eyes on the Street”: Pedestrian Density and Fear of Crime in Yaoundé, Cameroon," Sustainability, MDPI, vol. 12(13), pages 1-16, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Xiushi Yang, 2000. "Determinants of Migration Intentions in Hubei Province, China: Individual versus Family Migration," Environment and Planning A, , vol. 32(5), pages 769-787, May.
    2. Renard, Didier & Molenberghs, Geert & Geys, Helena, 2004. "A pairwise likelihood approach to estimation in multilevel probit models," Computational Statistics & Data Analysis, Elsevier, vol. 44(4), pages 649-667, January.
    3. repec:pri:crcwel:wp98-20-teitler is not listed on IDEAS
    4. Amoroso, S., 2013. "Heterogeneity of innovative, collaborative, and productive firm-level processes," Other publications TiSEM f5784a49-7053-401d-855d-1, Tilburg University, School of Economics and Management.
    5. Chun Wang & Steven W. Nydick, 2020. "On Longitudinal Item Response Theory Models: A Didactic," Journal of Educational and Behavioral Statistics, , vol. 45(3), pages 339-368, June.
    6. Gustavo Angeles & David K. Guilkey & Thomas A. Mroz, 2005. "The Impact of Community-Level Variables on Individual-Level Outcomes," Sociological Methods & Research, , vol. 34(1), pages 76-121, August.
    7. Anne Pebley & Noreen Goldman & Germán Rodríguez, 1996. "Prenatal and delivery care and childhood immunization in guatemala: Do family and community matter?," Demography, Springer;Population Association of America (PAA), vol. 33(2), pages 231-247, May.
    8. Mangham-Jefferies, Lindsay & Hanson, Kara & Mbacham, Wilfred & Onwujekwe, Obinna & Wiseman, Virginia, 2014. "What determines providers' stated preference for the treatment of uncomplicated malaria?," Social Science & Medicine, Elsevier, vol. 104(C), pages 98-106.
    9. Sun-Joo Cho & Paul Boeck & Susan Embretson & Sophia Rabe-Hesketh, 2014. "Additive Multilevel Item Structure Models with Random Residuals: Item Modeling for Explanation and Item Generation," Psychometrika, Springer;The Psychometric Society, vol. 79(1), pages 84-104, January.
    10. Fiona Steele & Ian Diamond & Duolao Wang, 1996. "The determinants of the duration of contraceptive use in China: A multilevel multinomial discrete-hazards mdeling approach," Demography, Springer;Population Association of America (PAA), vol. 33(1), pages 12-23, February.
    11. Patrali Chatterjee & Donna L. Hoffman & Thomas P. Novak, 2003. "Modeling the Clickstream: Implications for Web-Based Advertising Efforts," Marketing Science, INFORMS, vol. 22(4), pages 520-541, May.
    12. Diaz, Rafael E., 2007. "Comparison of PQL and Laplace 6 estimates of hierarchical linear models when comparing groups of small incident rates in cluster randomised trials," Computational Statistics & Data Analysis, Elsevier, vol. 51(6), pages 2871-2888, March.
    13. repec:jss:jstsof:08:i09 is not listed on IDEAS
    14. An, Xinming & Bentler, Peter M., 2012. "Efficient direct sampling MCEM algorithm for latent variable models with binary responses," Computational Statistics & Data Analysis, Elsevier, vol. 56(2), pages 231-244.
    15. Geraint Johnes & Robert McNabb, 2004. "Never Give up on the Good Times: Student Attrition in the UK," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 66(1), pages 23-47, February.
    16. Bu, Feifei, 2014. "Sibling configurations, educational aspiration and attainment," ISER Working Paper Series 2014-11, Institute for Social and Economic Research.
    17. Barbara Entwisle & Ronald Rindfuss & Stephen Walsh & Tom Evans & Sara Curran, 1997. "Geographic information systems, spatial network analysis, and contraceptive choice," Demography, Springer;Population Association of America (PAA), vol. 34(2), pages 171-187, May.
    18. Sturdivant, Rodney X. & Hosmer Jr., David W., 2007. "A smoothed residual based goodness-of-fit statistic for logistic hierarchical regression models," Computational Statistics & Data Analysis, Elsevier, vol. 51(8), pages 3898-3912, May.
    19. Shawn Kanaiaupuni & Katharine Donato, 1999. "Migradollars and mortality: The effects of migration on infant survival in Mexico," Demography, Springer;Population Association of America (PAA), vol. 36(3), pages 339-353, August.
    20. Baghishani, Hossein & Mohammadzadeh, Mohsen, 2011. "A data cloning algorithm for computing maximum likelihood estimates in spatial generalized linear mixed models," Computational Statistics & Data Analysis, Elsevier, vol. 55(4), pages 1748-1759, April.
    21. Dana A. Glei & Noreen Goldman & German Rodriguez, 2002. "Utilization of Care During Pregnancy in Rural Guatemala: Does Obstetrical Need Matters," Working Papers 308, Princeton University, Woodrow Wilson School of Public and International Affairs, Office of Population Research..
    22. Rebecca Sear & Fiona Steele & Ian McGregor & Ruth Mace, 2002. "The effects of kin on child mortality in rural gambia," Demography, Springer;Population Association of America (PAA), vol. 39(1), pages 43-63, February.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bpj:ijbist:v:6:y:2010:i:1:n:16. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Peter Golla (email available below). General contact details of provider: https://www.degruyter.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.