IDEAS home Printed from https://ideas.repec.org/a/bla/jorssa/v184y2021i4p1578-1598.html
   My bibliography  Save this article

Partially pooled propensity score models for average treatment effect estimation with multilevel data

Author

Listed:
  • Youjin Lee
  • Trang Q. Nguyen
  • Elizabeth A. Stuart

Abstract

Causal inference analyses often use existing observational data, which in many cases has some clustering of individuals. In this paper, we discuss propensity score weighting methods in a multilevel setting where within clusters individuals share unmeasured confounders that are related to treatment assignment and the potential outcomes. We focus in particular on settings where models with fixed cluster effects are either not feasible or not useful due to the presence of a large number of small clusters. We found, both through numerical experiments and theoretical derivations, that a strategy of grouping clusters with similar treatment prevalence and estimating propensity scores within such cluster groups is effective in reducing bias from unmeasured cluster‐level covariates under mild conditions on the outcome model. We apply our proposed method in evaluating the effectiveness of centre‐based pre‐school programme participation on children’s achievement at kindergarten, using the Early Childhood Longitudinal Study Kindergarten data.

Suggested Citation

  • Youjin Lee & Trang Q. Nguyen & Elizabeth A. Stuart, 2021. "Partially pooled propensity score models for average treatment effect estimation with multilevel data," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(4), pages 1578-1598, October.
  • Handle: RePEc:bla:jorssa:v:184:y:2021:i:4:p:1578-1598
    DOI: 10.1111/rssa.12741
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/rssa.12741
    Download Restriction: no

    File URL: https://libkey.io/10.1111/rssa.12741?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Arpino, Bruno & Mealli, Fabrizia, 2011. "The specification of the propensity score in multilevel observational studies," Computational Statistics & Data Analysis, Elsevier, vol. 55(4), pages 1770-1780, April.
    2. Sophia Rabe‐Hesketh & Anders Skrondal, 2006. "Multilevel modelling of complex survey data," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 169(4), pages 805-827, October.
    3. Hong, Guanglei & Raudenbush, Stephen W., 2006. "Evaluating Kindergarten Retention Policy: A Case Study of Causal Inference for Multilevel Observational Data," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 901-910, September.
    4. Nianbo Dong & Elizabeth A. Stuart & David Lenis & Trang Quynh Nguyen, 2020. "Using Propensity Score Analysis of Survey Data to Estimate Population Average Treatment Effects: A Case Study Comparing Different Methods," Evaluation Review, , vol. 44(1), pages 84-108, February.
    5. José R. Zubizarreta & Luke Keele, 2017. "Optimal Multilevel Matching in Clustered Observational Studies: A Case Study of the Effectiveness of Private Schools Under a Large-Scale Voucher System," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(518), pages 547-560, April.
    6. Loeb, Susanna & Bridges, Margaret & Bassok, Daphna & Fuller, Bruce & Rumberger, Russell W., 2007. "How much is too much? The influence of preschool centers on children's social and cognitive development," Economics of Education Review, Elsevier, vol. 26(1), pages 52-66, February.
    7. Skinner, Chris J. & D'Arrigo, Julia, 2011. "Inverse probability weighting for clustered nonresponse," LSE Research Online Documents on Economics 40308, London School of Economics and Political Science, LSE Library.
    8. C. J. Skinner & D'arrigo, 2011. "Inverse probability weighting for clustered nonresponse," Biometrika, Biometrika Trust, vol. 98(4), pages 953-966.
    9. Sneha Elango & Jorge Luis García & James J. Heckman & Andrés Hojman, 2015. "Early Childhood Education," NBER Chapters, in: Economics of Means-Tested Transfer Programs in the United States, Volume 2, pages 235-297, National Bureau of Economic Research, Inc.
    10. Carlson, S.A. & Fulton, J.E. & Lee, S.M. & Maynard, L.M. & Brown, D.R. & Kohl III, H.W. & Dietz, W.H., 2008. "Physical education and academic achievement in elementary school: Data from the early childhood longitudinal study," American Journal of Public Health, American Public Health Association, vol. 98(4), pages 721-727.
    11. Zeyu Xu & Charisse Gulosino, 2006. "How Does Teacher Quality Matter? The Effect of Teacher-Parent Partnership on Early Childhood Performance in Public and Private Schools," Education Economics, Taylor & Francis Journals, vol. 14(3), pages 345-367.
    12. Robert Thorndike, 1953. "Who belongs in the family?," Psychometrika, Springer;The Psychometric Society, vol. 18(4), pages 267-276, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Youmi Suk, 2024. "A Within-Group Approach to Ensemble Machine Learning Methods for Causal Inference in Multilevel Studies," Journal of Educational and Behavioral Statistics, , vol. 49(1), pages 61-91, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Plewis Ian & Shlomo Natalie, 2017. "Using Response Propensity Models to Improve the Quality of Response Data in Longitudinal Studies," Journal of Official Statistics, Sciendo, vol. 33(3), pages 753-779, September.
    2. Jouni Kuha & Myrsini Katsikatsou & Irini Moustaki, 2018. "Latent variable modelling with non‐ignorable item non‐response: multigroup response propensity models for cross‐national analysis," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 181(4), pages 1169-1192, October.
    3. van Huizen, Thomas & Plantenga, Janneke, 2018. "Do children benefit from universal early childhood education and care? A meta-analysis of evidence from natural experiments," Economics of Education Review, Elsevier, vol. 66(C), pages 206-222.
    4. K. C. Culver & Nathaniel Bray & John Braxton, 2024. "On My Honor: A Quasi-Experimental Analysis of Honors Students’ Perceptions of Workload and Cognitive Challenge," Research in Higher Education, Springer;Association for Institutional Research, vol. 65(4), pages 679-704, June.
    5. Mary Ying-Fang Wang & Paul Tuss & Lihong Qi, 2019. "Augmented Weighted Estimators Dealing with Practical Positivity Violation to Causal inferences in a Random Coefficient Model," Psychometrika, Springer;The Psychometric Society, vol. 84(2), pages 447-467, June.
    6. Helene Boistard & Guillaume Chauvet & David Haziza, 2016. "Doubly Robust Inference for the Distribution Function in the Presence of Missing Survey Data," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 43(3), pages 683-699, September.
    7. Brick J. Michael, 2013. "Unit Nonresponse and Weighting Adjustments: A Critical Review," Journal of Official Statistics, Sciendo, vol. 29(3), pages 329-353, June.
    8. Md Saiful Islam & Md Sarowar Morshed & Gary J Young & Md Noor-E-Alam, 2019. "Robust policy evaluation from large-scale observational studies," PLOS ONE, Public Library of Science, vol. 14(10), pages 1-19, October.
    9. Li He & Yu-Bo Wang & William C. Bridges & Zhulin He & S. Megan Che, 2023. "Bayesian Framework for Causal Inference with Principal Stratification and Clusters," Statistics in Biosciences, Springer;International Chinese Statistical Association, vol. 15(1), pages 114-140, April.
    10. K. C. Culver & Nicholas A. Bowman, 2020. "Is What Glitters Really Gold? A Quasi-Experimental Study of First-Year Seminars and College Student Success," Research in Higher Education, Springer;Association for Institutional Research, vol. 61(2), pages 167-196, March.
    11. Youmi Suk & Hyunseung Kang, 2022. "Robust Machine Learning for Treatment Effects in Multilevel Observational Studies Under Cluster-level Unmeasured Confounding," Psychometrika, Springer;The Psychometric Society, vol. 87(1), pages 310-343, March.
    12. Aleksey Oshchepkov & Anna Shirokanova, 2020. "Multilevel Modeling For Economists: Why, When And How," HSE Working papers WP BRP 233/EC/2020, National Research University Higher School of Economics.
    13. Hans Walter Steinhauer & Christian Aßmann & Sabine Zinn & Solange Goßmann & Susanne Rässler, 2015. "Sampling and Weighting Cohort Samples in Institutional Contexts," AStA Wirtschafts- und Sozialstatistisches Archiv, Springer;Deutsche Statistische Gesellschaft - German Statistical Society, vol. 9(2), pages 131-157, November.
    14. Koecklin, Manuel Tong & Longoria, Genaro & Fitiwi, Desta Z. & DeCarolis, Joseph F. & Curtis, John, 2021. "Public acceptance of renewable electricity generation and transmission network developments: Insights from Ireland," Energy Policy, Elsevier, vol. 151(C).
    15. Aleksandra Parteka & Joanna Wolszczak-Derlacz, 2020. "Wage response to global production links: evidence for workers from 28 European countries (2005–2014)," Review of World Economics (Weltwirtschaftliches Archiv), Springer;Institut für Weltwirtschaft (Kiel Institute for the World Economy), vol. 156(4), pages 769-801, November.
    16. Per Strömblad & Gunnar Myrberg, 2013. "Urban Inequality and Political Recruitment," Urban Studies, Urban Studies Journal Limited, vol. 50(5), pages 1049-1065, April.
    17. Robert Rogers & Doan Hai Ma & Tra Nguyen & Ngoc Anh Nguyen, 2019. "Early childhood education and cognitive outcomes in adolescence: a longitudinal study from Vietnam," Education Economics, Taylor & Francis Journals, vol. 27(6), pages 658-669, November.
    18. Apps, Patricia & Mendolia, Silvia & Walker, Ian, 2013. "The impact of pre-school on adolescents’ outcomes: Evidence from a recent English cohort," Economics of Education Review, Elsevier, vol. 37(C), pages 183-199.
    19. Becken, Susanne & Stantic, Bela & Chen, Jinyan & Connolly, Rod M., 2022. "Twitter conversations reveal issue salience of aviation in the broader context of climate change," Journal of Air Transport Management, Elsevier, vol. 98(C).
    20. Adrià Muntaner-Mas & Josep Vidal-Conti & Jo Salmon & Pere Palou-Sampol, 2020. "Associations of Heart Rate Measures during Physical Education with Academic Performance and Executive Function in Children: A Cross-Sectional Study," IJERPH, MDPI, vol. 17(12), pages 1-12, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssa:v:184:y:2021:i:4:p:1578-1598. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/rssssea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.