IDEAS home Printed from https://ideas.repec.org/a/sae/somere/v47y2018i3p430-457.html
   My bibliography  Save this article

Design Effects of Multilevel Estimates From National Probability Samples

Author

Listed:
  • Laura M. Stapleton
  • Yoonjeong Kang

Abstract

This research empirically evaluates data sets from the National Center for Education Statistics (NCES) for design effects of ignoring the sampling design in weighted two-level analyses. Currently, researchers may ignore the sampling design beyond the levels that they model which might result in incorrect inferences regarding hypotheses due to biased standard error estimates; the degree of bias depends on the informativeness of any ignored stratification and clustering in the sampling design. Some multilevel software packages accommodate first-stage sampling design information for two-level models but not all. For five example public release data sets from the NCES, design effects of ignoring the sampling design in unconditional and conditional two-level models are presented for 15 dependent variables selected based on a review of published research using these five data sets. Empirical findings suggest that there are minor effects of ignoring the additional sampling design and no differences in inference would be made had the first-stage sampling design been ignored. Strategically, researchers without access to multilevel software that can accommodate the sampling might consider including stratification variables as independent variables at level 2 of their model.

Suggested Citation

  • Laura M. Stapleton & Yoonjeong Kang, 2018. "Design Effects of Multilevel Estimates From National Probability Samples," Sociological Methods & Research, , vol. 47(3), pages 430-457, August.
    • Handle: RePEc:sae:somere:v:47:y:2018:i:3:p:430-457
    DOI: 10.1177/0049124116630563
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0049124116630563
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0049124116630563?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Sophia Rabe‐Hesketh & Anders Skrondal, 2006. "Multilevel modelling of complex survey data," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 169(4), pages 805-827, October.
    2. Sophia Rabe-Hesketh & Anders Skrondal & Andrew Pickles, 2004. "Generalized multilevel structural equation modeling," Psychometrika, Springer;The Psychometric Society, vol. 69(2), pages 167-190, June.
    3. D. Pfeffermann & C. J. Skinner & D. J. Holmes & H. Goldstein & J. Rasbash, 1998. "Weighting for unequal selection probabilities in multilevel models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 60(1), pages 23-40.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Woojin Chung & Roeul Kim, 2020. "A Reversal of the Association between Education Level and Obesity Risk during Ageing: A Gender-Specific Longitudinal Study in South Korea," IJERPH, MDPI, vol. 17(18), pages 1-19, September.
    2. Patricia Dörr & Jan Pablo Burgard, 2019. "Data-driven transformations and survey-weighting for linear mixed models," Research Papers in Economics 2019-16, University of Trier, Department of Economics.
    3. Arrieta-Paredes, Mary-Paz & Hallsworth, Alan G. & Coca-Stefaniak, J. Andres, 2020. "Small shop survival – The financial response to a global financial crisis," Journal of Retailing and Consumer Services, Elsevier, vol. 53(C).
    4. Joseph L Dieleman & Tara Templin, 2014. "Random-Effects, Fixed-Effects and the within-between Specification for Clustered Data in Observational Health Studies: A Simulation Study," PLOS ONE, Public Library of Science, vol. 9(10), pages 1-17, October.
    5. Woojin Chung & Roeul Kim, 2020. "Differential Risk of Cognitive Impairment across Paid and Unpaid Occupations in the Middle-Age Population: Evidence from the Korean Longitudinal Study of Aging, 2006–2016," IJERPH, MDPI, vol. 17(9), pages 1-14, April.
    6. Michele Battisti & Andrea Mario Lavezzi & Lucio Masserini & Monica Pratesi, 2018. "Resisting the extortion racket: an empirical analysis," European Journal of Law and Economics, Springer, vol. 46(1), pages 1-37, August.
    7. Woojin Chung & Roeul Kim, 2020. "Which Occupation is Highly Associated with Cognitive Impairment? A Gender-Specific Longitudinal Study of Paid and Unpaid Occupations in South Korea," IJERPH, MDPI, vol. 17(21), pages 1-17, October.
    8. Nora Würz & Timo Schmid & Nikos Tzavidis, 2022. "Estimating regional income indicators under transformations and access to limited population auxiliary information," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 185(4), pages 1679-1706, October.
    9. Anders Skrondal & Sophia Rabe‐Hesketh, 2009. "Prediction in multilevel generalized linear models," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 172(3), pages 659-687, June.
    10. Sophia Rabe‐Hesketh & Anders Skrondal, 2006. "Multilevel modelling of complex survey data," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 169(4), pages 805-827, October.
    11. Robert G. Clark & David G. Steel, 2022. "Sample design for analysis using high‐influence probability sampling," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 185(4), pages 1733-1756, October.
    12. Francesco Schirripa Spagnolo & Nicola Salvati & Antonella D’Agostino & Ides Nicaise, 2020. "The use of sampling weights in M‐quantile random‐effects regression: an application to Programme for International Student Assessment mathematics scores," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 69(4), pages 991-1012, August.
    13. Bowen, Mary Elizabeth, 2009. "Childhood socioeconomic status and racial differences in disability: Evidence from the Health and Retirement Study (1998-2006)," Social Science & Medicine, Elsevier, vol. 69(3), pages 433-441, August.
    14. Ana Maria Osorio & Catalina Bolancé & Nyovane Madise & Katharina Rathmann, 2013. "Social Determinants of Child Health in Colombia: Can Community Education Moderate the Effect of Family Characteristics?," Working Papers XREAP2013-02, Xarxa de Referència en Economia Aplicada (XREAP), revised Mar 2013.
    15. Diana M. Hechavarría & Siri A. Terjesen & Amy E. Ingram & Maija Renko & Rachida Justo & Amanda Elam, 2017. "Taking care of business: the impact of culture and gender on entrepreneurs’ blended value creation goals," Small Business Economics, Springer, vol. 48(1), pages 225-257, January.
    16. Amini, Chiara & Nivorozhkin, Eugene, 2015. "The urban–rural divide in educational outcomes: Evidence from Russia," International Journal of Educational Development, Elsevier, vol. 44(C), pages 118-133.
    17. Yergeau, Marie-Eve, 2020. "Tourism and local welfare: A multilevel analysis in Nepal’s protected areas," World Development, Elsevier, vol. 127(C).
    18. Glen McGee & Jonathan Schildcrout & Sharon‐Lise Normand & Sebastien Haneuse, 2020. "Outcome‐dependent sampling in cluster‐correlated data settings with application to hospital profiling," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 183(1), pages 379-402, January.
    19. Oǧuz-Alper, Melike & Berger, Yves G., 2020. "Modelling multilevel data under complex sampling designs: An empirical likelihood approach," Computational Statistics & Data Analysis, Elsevier, vol. 145(C).
    20. Sophia Rabe-Hesketh & Anders Skrondal, 2007. "Multilevel and Latent Variable Modeling with Composite Links and Exploded Likelihoods," Psychometrika, Springer;The Psychometric Society, vol. 72(2), pages 123-140, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:somere:v:47:y:2018:i:3:p:430-457. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.