IDEAS home Printed from https://ideas.repec.org/a/zbw/espost/232505.html
   My bibliography  Save this article

Investigating the impact of interventions on educational disparities: Estimating average treatment effects (ATEs) is not sufficient

Author

Listed:
  • Pietrzyk, Irena
  • Erdmann, Melinda

Abstract

In most national education systems, educational attainment depends heavily on parental socioeconomic status, which challenges the notion of equal opportunity. Educational interventions have increasingly become the focus of public and scientific debate as a tool for mitigating these disparities. Establishing appropriate methods for investigating the impact of these programs on inequality is of great interest to researchers, policymakers and practitioners. We help make methodological refinements in this research area. In defining educational disparities as unequal chances for educational attainment of different social groups on the macro level, we suggest considering not only how strongly participating persons from distinct social groups benefit from a program (i.e., group-specific ATEs) but also how many persons from distinct social groups take part in the program under real world conditions (i.e., group-specific participation rates (PRs)). As we define the PR as the share of a subpopulation participating in a program under real world conditions, PRs link the real world and macro level to the intervention level, and therefore, its consideration confers external validity. We develop a formula of how group-specific ATEs and PRs jointly contribute to an intervention’s effect on disparities, and we simulate their joint contributions to disparities in university enrollment within reasonable limits and by presenting a fictitious yet realistic example. Because the contributions of group-specific ATEs and PRs to disparities have not been formalized yet, our results underscore the essential importance of PRs for understanding the impact of educational interventions on disparities. Furthermore, the illustration helps to correct the misconception that focusing exclusively on group-specific ATEs (or on group-specific PRs) is sensible when drawing conclusions about a program’s effect on disparities. More specifically, interventions that appear to mitigate disparities based on a comparison of group-specific ATEs might in fact exacerbate inequality if low-SES persons are underrepresented in the program. We close with recommendations for future research and a review of the study’s limitations.

Suggested Citation

  • Pietrzyk, Irena & Erdmann, Melinda, 2020. "Investigating the impact of interventions on educational disparities: Estimating average treatment effects (ATEs) is not sufficient," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 65, pages 1-1.
    • Handle: RePEc:zbw:espost:232505
    as

    Download full text from publisher

    File URL: https://www.econstor.eu/bitstream/10419/232505/1/Full-text-article-Pietrzyk-et-al-Investigating-the-impact.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Saifuddin Ahmed & Andreea A Creanga & Duff G Gillespie & Amy O Tsui, 2010. "Economic Status, Education and Empowerment: Implications for Maternal Health Service Utilization in Developing Countries," PLOS ONE, Public Library of Science, vol. 5(6), pages 1-6, June.
    2. repec:mpr:mprres:4029 is not listed on IDEAS
    3. Neil S. Seftor & Arif Mamun & Allen Schirm, "undated". "The Impacts of Regular Upward Bound on Postsecondary Outcomes 7-9 Years After Scheduled High School Graduation," Mathematica Policy Research Reports 2ee3fcef720a4257a79beeea2, Mathematica Policy Research.
    4. Christopher Avery, 2013. "Evaluation of the College Possible Program: Results from a Randomized Controlled Trial," NBER Working Papers 19562, National Bureau of Economic Research, Inc.
    5. Ehlert, Martin & Finger, Claudia & Rusconi, Alessandra & Solga, Heike, 2017. "Applying to college: Do information deficits lower the likelihood of college-eligible students from less-privileged families to pursue their college intentions?: Evidence from a field experiment," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 67, pages 193-212.
    6. David Myers & Robert B. Olsen & Neil Seftor & Julie Young & Christina Clark Tuttle, "undated". "The Impacts of Regular Upward Bound: Results from the Third Follow-Up Data Collection," Mathematica Policy Research Reports 145dc84be33e47e494cb5569f, Mathematica Policy Research.
    7. repec:hal:spmain:info:hdl:2441/527ht1a96e837pq2dubgo2953q is not listed on IDEAS
    8. Justine Hastings & Christopher A. Neilson & Seth D. Zimmerman, 2015. "The Effects of Earnings Disclosure on College Enrollment Decisions," NBER Working Papers 21300, National Bureau of Economic Research, Inc.
    9. Loyalka, Prashant & Song, Yingquan & Wei, Jianguo & Zhong, Weiping & Rozelle, Scott, 2013. "Information, college decisions and financial aid: Evidence from a cluster-randomized controlled trial in China," Economics of Education Review, Elsevier, vol. 36(C), pages 26-40.
    10. Deaton, Angus & Cartwright, Nancy, 2018. "Understanding and misunderstanding randomized controlled trials," Social Science & Medicine, Elsevier, vol. 210(C), pages 2-21.
    11. Andreas Vilhelmsson & Per-Olof Östergren, 2018. "Reducing health inequalities with interventions targeting behavioral factors among individuals with low levels of education - A rapid review," PLOS ONE, Public Library of Science, vol. 13(4), pages 1-14, April.
    12. Martin McGuigan & Sandra McNally & Gill Wyness, 2016. "Student Awareness of Costs and Benefits of Educational Decisions: Effects of an Information Campaign," Journal of Human Capital, University of Chicago Press, vol. 10(4), pages 482-519.
    13. Bonilla-Mejía, Leonardo & Bottan, Nicolas L. & Ham, Andrés, 2019. "Information policies and higher education choices experimental evidence from Colombia," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 83(C).
    14. Estelle Herbaut & Koen Geven, 2019. "What Works to Reduce Inequalities in Higher Education? A Systematic Review of the (Quasi-)Experimental Literature on Outreach and Financial Aid," Working Papers hal-03456943, HAL.
    15. Scott Carrell & Bruce Sacerdote, 2017. "Why Do College-Going Interventions Work?," American Economic Journal: Applied Economics, American Economic Association, vol. 9(3), pages 124-151, July.
    16. repec:mpr:mprres:6088 is not listed on IDEAS
    17. Donald B. Rubin, 2005. "Causal Inference Using Potential Outcomes: Design, Modeling, Decisions," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 322-331, March.
    18. Herbaut,Estelle & Geven,Koen Martijn, 2019. "What Works to Reduce Inequalities in Higher Education ? A Systematic Review of the (Quasi-)Experimental Literature on Outreach and Financial Aid," Policy Research Working Paper Series 8802, The World Bank.
    19. Benjamin L. Castleman & Lindsay C. Page & Korynn Schooley, 2014. "The Forgotten Summer: Does the Offer of College Counseling After High School Mitigate Summer Melt Among College‐Intending, Low‐Income High School Graduates?," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 33(2), pages 320-344, March.
    20. Eric P. Bettinger & Bridget Terry Long & Philip Oreopoulos & Lisa Sanbonmatsu, 2012. "The Role of Application Assistance and Information in College Decisions: Results from the H&R Block Fafsa Experiment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 127(3), pages 1205-1242.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Erdmann, Melinda & Pietrzyk, Irena Magdalena & Schneider, Juliana & Helbig, Marcel & Jacob, Marita & Allmendinger, Jutta, 2022. "Bildungsungleichheit nach der Hochschulreife - das lässt sich ändern: Eine Untersuchung der Wirksamkeit eines intensiven Beratungsprogramms 1,5 Jahre nach dem Abitur," Discussion Papers, Presidential Department P 2022-002, WZB Berlin Social Science Center.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. repec:hal:spmain:info:hdl:2441/527ht1a96e837pq2dubgo2953q is not listed on IDEAS
    2. repec:spo:wpmain:info:hdl:2441/527ht1a96e837pq2dubgo2953q is not listed on IDEAS
    3. Estelle Herbaut & Koen Geven, 2019. "What Works to Reduce Inequalities in Higher Education? A Systematic Review of the (Quasi-)Experimental Literature on Outreach and Financial Aid," SciencePo Working papers Main hal-03456943, HAL.
    4. Estelle Herbaut & Koen Geven, 2019. "What Works to Reduce Inequalities in Higher Education? A Systematic Review of the (Quasi-)Experimental Literature on Outreach and Financial Aid," Working Papers hal-03456943, HAL.
    5. Damgaard, Mette Trier & Nielsen, Helena Skyt, 2018. "Nudging in education," Economics of Education Review, Elsevier, vol. 64(C), pages 313-342.
    6. Peter, Frauke H. & Zambre, Vaishali, 2017. "Intended college enrollment and educational inequality: Do students lack information?," Economics of Education Review, Elsevier, vol. 60(C), pages 125-141.
    7. Avitabile, Ciro & de Hoyos, Rafael, 2018. "The heterogeneous effect of information on student performance: Evidence from a randomized control trial in Mexico," Journal of Development Economics, Elsevier, vol. 135(C), pages 318-348.
    8. Henning Hermes & Philipp Lergetporer & Frauke Peter & Simon Wiederhold, 2024. "Application Barriers and the Socioeconomic Gap in Child Care Enrollment," Working Papers 2024-022, Human Capital and Economic Opportunity Working Group.
    9. Henning Hermes & Philipp Lergetporer & Frauke Peter & Simon Wiederhold, 2021. "Application Barriers and the Socioeconomic Gap in Child Care Enrollment," CESifo Working Paper Series 9282, CESifo.
    10. Ballarino, Gabriele & Filippin, Antonio & Abbiati, Giovanni & Argentin, Gianluca & Barone, Carlo & Schizzerotto, Antonio, 2022. "The effects of an information campaign beyond university enrolment: A large-scale field experiment on the choices of high school students," Economics of Education Review, Elsevier, vol. 91(C).
    11. Frauke H. Peter & C. Katharina Spieß & Vaishali Zambre, 2018. "Informing Students about College: An Efficient Way to Decrease the Socio-Economic Gap in Enrollment: Evidence from a Randomized Field Experiment," Discussion Papers of DIW Berlin 1770, DIW Berlin, German Institute for Economic Research.
    12. French, Robert & Oreopoulos, Philip, 2017. "Behavioral barriers transitioning to college," Labour Economics, Elsevier, vol. 47(C), pages 48-63.
    13. Bonilla-Mejía, Leonardo & Bottan, Nicolas L. & Ham, Andrés, 2019. "Information policies and higher education choices experimental evidence from Colombia," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 83(C).
    14. Adam M. Lavecchia & Heidi Liu & Philip Oreopoulos, 2014. "Behavioral Economics of Education: Progress and Possibilities," NBER Working Papers 20609, National Bureau of Economic Research, Inc.
    15. Peter, Frauke & Spiess, C. Katharina & Zambre, Vaishali, 2021. "Informing students about college: Increasing enrollment using a behavioral intervention?," Journal of Economic Behavior & Organization, Elsevier, vol. 190(C), pages 524-549.
    16. Herber, Stefanie P., 2018. "The role of information in the application for highly selective scholarships: Evidence from a randomized field experiment," Economics of Education Review, Elsevier, vol. 62(C), pages 287-301.
    17. Christopher Neilson & Adam Altmejd & Andres Barrios-Fernandez & Marin Drlje & Dejan Kovac, 2019. "Siblings' Effects on College and Major Choices: Evidence from Chile, Croatia and Sweden," Working Papers 633, Princeton University, Department of Economics, Industrial Relations Section..
    18. Adam Altmejd & Andrés Barrios-Fernández & Marin Drlje & Joshua Goodman & Michael Hurwitz & Dejan Kovac & Christine Mulhern & Christopher Neilson & Jonathan Smith, 2021. "O Brother, Where Start Thou? Sibling Spillovers on College and Major Choice in Four Countries," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 136(3), pages 1831-1886.
    19. Joshua Goodman & Michael Hurwitz & Jonathan Smith, "undated". "Access to Four-Year Public Colleges and Degree Completion," Working Paper 175226, Harvard University OpenScholar.
    20. Bleemer, Zachary & Zafar, Basit, 2018. "Intended college attendance: Evidence from an experiment on college returns and costs," Journal of Public Economics, Elsevier, vol. 157(C), pages 184-211.
    21. Hermes, Henning & Lergetporer, Philipp & Peter, Frauke & Wiederhold, Simon, 2021. "Behavioral Barriers and the Socioeconomic Gap in Child Care Enrollment," IZA Discussion Papers 14698, Institute of Labor Economics (IZA).
    22. Christopher Erwin & Melissa Binder & Cynthia Miller & Kate Krause, 2020. "Performance-based aid, enhanced advising, and the income gap in college graduation: Evidence from a randomized controlled trial," Working Papers 2020-06, Auckland University of Technology, Department of Economics.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zbw:espost:232505. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ZBW - Leibniz Information Centre for Economics (email available below). General contact details of provider: https://edirc.repec.org/data/zbwkide.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.