IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/28205.html
   My bibliography  Save this paper

School’s Out: Experimental Evidence on Limiting Learning Loss Using “Low-Tech” in a Pandemic

Author

Listed:
  • Noam Angrist
  • Peter Bergman
  • Moitshepi Matsheng

Abstract

Schools closed extensively during the COVID-19 pandemic and occur in other settings, such as teacher strikes and natural disasters. This paper provides some of the first experimental evidence on strategies to minimize learning loss when schools close. We run a randomized trial of low-technology interventions – SMS messages and phone calls – with parents to support their child. The combined treatment cost-effectively improves learning by 0.12 standard deviations. We develop remote assessment innovations, which show robust learning outcomes. Our findings have immediate policy relevance and long-run implications for the role of technology and parents as partial educational substitutes when schooling is disrupted.

Suggested Citation

  • Noam Angrist & Peter Bergman & Moitshepi Matsheng, 2020. "School’s Out: Experimental Evidence on Limiting Learning Loss Using “Low-Tech” in a Pandemic," NBER Working Papers 28205, National Bureau of Economic Research, Inc.
  • Handle: RePEc:nbr:nberwo:28205
    Note: CH DEV ED IO
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w28205.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. World Bank, 2018. "Information and Communications for Development 2018," World Bank Publications - Books, The World Bank Group, number 30437.
    2. Noam Angrist & Peter Bergman & David K. Evans & Susannah Hares & Matthew C. H. Jukes & Thato Letsomo, 2020. "Practical Lessons for Phone-Based Assessments of Learning," Working Papers 534, Center for Global Development, revised 10 Jul 2020.
    3. Abhijit Banerjee & Rukmini Banerji & James Berry & Esther Duflo & Harini Kannan & Shobhini Mukerji & Marc Shotland & Michael Walton, 2017. "From Proof of Concept to Scalable Policies: Challenges and Solutions, with an Application," Journal of Economic Perspectives, American Economic Association, vol. 31(4), pages 73-102, Fall.
    4. Siebert, W. Stanley & Wei, Xiangdong & Wong, Ho Lun & Zhou, Xiang, 2018. "Student Feedback, Parent-Teacher Communication, and Academic Performance: Experimental Evidence from Rural China," IZA Discussion Papers 11347, Institute of Labor Economics (IZA).
    5. Abhijit V. Banerjee & Shawn Cole & Esther Duflo & Leigh Linden, 2007. "Remedying Education: Evidence from Two Randomized Experiments in India," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 122(3), pages 1235-1264.
    6. David Jaume & Alexander Willén, 2019. "The Long-Run Effects of Teacher Strikes: Evidence from Argentina," Journal of Labor Economics, University of Chicago Press, vol. 37(4), pages 1097-1139.
    7. Karthik Muralidharan & Abhijeet Singh & Alejandro J. Ganimian, 2019. "Disrupting Education? Experimental Evidence on Technology-Aided Instruction in India," American Economic Review, American Economic Association, vol. 109(4), pages 1426-1460, April.
    8. Gallego, Francisco A. & Malamud, Ofer & Pop-Eleches, Cristian, 2020. "Parental monitoring and children's internet use: The role of information, control, and cues," Journal of Public Economics, Elsevier, vol. 188(C).
    9. Uri Gneezy & John A. List & Jeffrey A. Livingston & Xiangdong Qin & Sally Sadoff & Yang Xu, 2019. "Measuring Success in Education: The Role of Effort on the Test Itself," American Economic Review: Insights, American Economic Association, vol. 1(3), pages 291-308, December.
    10. Jonathan Guryan & Jens Ludwig & Monica P. Bhatt & Philip J. Cook & Jonathan M. V. Davis & Kenneth Dodge & George Farkas & Roland G. Fryer Jr. & Susan Mayer & Harold Pollack & Laurence Steinberg & Greg, 2023. "Not Too Late: Improving Academic Outcomes among Adolescents," American Economic Review, American Economic Association, vol. 113(3), pages 738-765, March.
    11. Esther Duflo & Pascaline Dupas & Michael Kremer, 2011. "Peer Effects, Teacher Incentives, and the Impact of Tracking: Evidence from a Randomized Evaluation in Kenya," American Economic Review, American Economic Association, vol. 101(5), pages 1739-1774, August.
    12. Eric A. Hanushek & Ludger Woessmann, 2020. "The economic impacts of learning losses," OECD Education Working Papers 225, OECD Publishing.
    13. Angrist,Noam & Djankov,Simeon & Goldberg,Pinelopi Koujianou & Patrinos,Harry Anthony, 2019. "Measuring Human Capital," Policy Research Working Paper Series 8742, The World Bank.
    14. Damien de Walque & Christine Valente, 2023. "Incentivizing School Attendance in the Presence of Parent-Child Information Frictions," American Economic Journal: Economic Policy, American Economic Association, vol. 15(3), pages 256-285, August.
    15. Kraft, Matthew A. & Rogers, Todd, 2015. "The underutilized potential of teacher-to-parent communication: Evidence from a field experiment," Economics of Education Review, Elsevier, vol. 47(C), pages 49-63.
    16. Abhijit V. Banerjee & Rukmini Banerji & Esther Duflo & Rachel Glennerster & Stuti Khemani, 2010. "Pitfalls of Participatory Programs: Evidence from a Randomized Evaluation in Education in India," American Economic Journal: Economic Policy, American Economic Association, vol. 2(1), pages 1-30, February.
    17. Annie Duflo & Jessica Kiessel & Adrienne Lucas, 2020. "Experimental Evidence on Alternative Policies to Increase Learning at Scale," NBER Working Papers 27298, National Bureau of Economic Research, Inc.
    18. Dinkelman, Taryn & Berlinski, Samuel & Busso, Matias & Martinez A., Claudia, 2021. "Reducing Parent-School Information Gaps and Improving Education Outcomes: Evidence from High-Frequency Text Messages," CEPR Discussion Papers 15949, C.E.P.R. Discussion Papers.
    19. Eric Bettinger & Robert Fairlie & Anastasia Kapuza & Elena Kardanova & Prashant Loyalka & Andrey Zakharov, 2023. "Diminishing Marginal Returns to Computer‐Assisted Learning," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 42(2), pages 552-570, March.
    20. Bacher-Hicks, Andrew & Goodman, Joshua & Mulhern, Christine, 2021. "Inequality in household adaptation to schooling shocks: Covid-induced online learning engagement in real time," Journal of Public Economics, Elsevier, vol. 193(C).
    21. Angrist,Noam & Evans,David & Filmer,Deon P. & Glennerster,Rachel & Rogers,F. Halsey & Sabarwal,Shwetlena, 2020. "How to Improve Education Outcomes Most Efficiently ? A Comparison of 150 Interventions Using theNew Learning-Adjusted Years of Schooling Metric," Policy Research Working Paper Series 9450, The World Bank.
    22. Rebecca Dizon-Ross, 2019. "Parents' Beliefs about Their Children's Academic Ability: Implications for Educational Investments," American Economic Review, American Economic Association, vol. 109(8), pages 2728-2765, August.
    23. Jenny C. Aker & Christopher Ksoll & Travis J. Lybbert, 2012. "Can Mobile Phones Improve Learning? Evidence from a Field Experiment in Niger," American Economic Journal: Applied Economics, American Economic Association, vol. 4(4), pages 94-120, October.
    24. Jenny C. Aker & Christopher Ksoll, 2020. "Can ABC Lead to Sustained 123? The Medium-Term Effects of a Technology-Enhanced Adult Education Program," Economic Development and Cultural Change, University of Chicago Press, vol. 68(3), pages 1081-1102.
    25. Christopher Doss & Erin M. Fahle & Susanna Loeb & Benjamin N. York, 2019. "More Than Just a Nudge: Supporting Kindergarten Parents with Differentiated and Personalized Text Messages," Journal of Human Resources, University of Wisconsin Press, vol. 54(3), pages 567-603.
    26. João Pedro & Amer Hasan & Diana Goldemberg & Koen Geven & Syedah Aroob Iqbal, 2021. "Simulating the Potential Impacts of COVID-19 School Closures on Schooling and Learning Outcomes: A Set of Global Estimates [Tackling Inequity in Education during and after COVID-19]," The World Bank Research Observer, World Bank, vol. 36(1), pages 1-40.
    27. Rema Hanna & Sendhil Mullainathan & Joshua Schwartzstein, 2014. "Learning Through Noticing: Theory and Evidence from a Field Experiment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 129(3), pages 1311-1353.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Carlana, Michela & La Ferrara, Eliana, 2021. "Apart but Connected: Online Tutoring and Student Outcomes during the COVID-19 Pandemic," IZA Discussion Papers 14094, Institute of Labor Economics (IZA).
    2. Dinkelman, Taryn & Berlinski, Samuel & Busso, Matias & Martinez A., Claudia, 2021. "Reducing Parent-School Information Gaps and Improving Education Outcomes: Evidence from High-Frequency Text Messages," CEPR Discussion Papers 15949, C.E.P.R. Discussion Papers.
    3. M. Niaz Asadullah & Anindita Bhattacharjee, 2022. "Digital Divide or Digital Provide? Technology, Time Use, and Learning Loss during COVID-19," Journal of Development Studies, Taylor & Francis Journals, vol. 58(10), pages 1934-1957, October.
    4. Hugues Champeaux & Lucia Mangiavacchi & Francesca Marchetta & Luca Piccoli, 2022. "Child development and distance learning in the age of COVID-19," Review of Economics of the Household, Springer, vol. 20(3), pages 659-685, September.
    5. Hashibul Hassan & Asad Islam & Abu Siddique & Liang Choon Wang, 2024. "Telementoring and Homeschooling During School Closures: a Randomised Experiment in Rural Bangladesh," The Economic Journal, Royal Economic Society, vol. 134(662), pages 2418-2438.
    6. Anastasia, Giacomo & Boeri, Tito & Kudlyak, Marianna & Zholud, Oleksandr, 2022. "The Labor Market in Ukraine: Rebuild Better," CEPR Discussion Papers 17769, C.E.P.R. Discussion Papers.
    7. Crawfurd, Lee & Evans, David K. & Hares, Susannah & Sandefur, Justin, 2023. "Live tutoring calls did not improve learning during the COVID-19 pandemic in Sierra Leone," Journal of Development Economics, Elsevier, vol. 164(C).
    8. Allen, James & Mahumane, Arlete & Riddell, James & Rosenblat, Tanya & Yang, Dean & Yu, Hang, 2022. "Teaching and incentives: Substitutes or complements?," Economics of Education Review, Elsevier, vol. 91(C).
    9. Lafortune, Jeanne & Pugatch, Todd & Tessada, José & Ubfal, Diego, 2022. "Can interactive online training make high school students more entrepreneurial? Experimental evidence from Rwanda," GLO Discussion Paper Series 1041, Global Labor Organization (GLO).
    10. Amer-Mestre, Josep & Ayarza-Astigarraga, Alaitz & Lopes, Marta C, 2021. "E-Learning Engagement Gap during School Closures: Differences by Academic Performance," IZA Discussion Papers 14904, Institute of Labor Economics (IZA).
    11. Clare Halloran & Rebecca Jack & James C. Okun & Emily Oster, 2021. "Pandemic Schooling Mode and Student Test Scores: Evidence from US States," NBER Working Papers 29497, National Bureau of Economic Research, Inc.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Noam Angrist & Peter Bergman & Moitshepi Matsheng, 2022. "Experimental evidence on learning using low-tech when school is out," Nature Human Behaviour, Nature, vol. 6(7), pages 941-950, July.
    2. Hashibul Hassan & Asad Islam & Abu Siddique & Liang Choon Wang, 2024. "Telementoring and Homeschooling During School Closures: a Randomised Experiment in Rural Bangladesh," The Economic Journal, Royal Economic Society, vol. 134(662), pages 2418-2438.
    3. Angrist, Noam & de Barros, Andreas & Bhula, Radhika & Chakera, Shiraz & Cummiskey, Chris & DeStefano, Joseph & Floretta, John & Kaffenberger, Michelle & Piper, Benjamin & Stern, Jonathan, 2021. "Building back better to avert a learning catastrophe: Estimating learning loss from COVID-19 school shutdowns in Africa and facilitating short-term and long-term learning recovery," International Journal of Educational Development, Elsevier, vol. 84(C).
    4. Emily Beam & Priya Mukherjee & Laia Navarro-Sola, 2022. "Lowering Barriers to Remote Education: Experimental Impacts on Parental Responses and Learning," Working Papers 2022-030, Human Capital and Economic Opportunity Working Group.
    5. Sabrin Beg & Waqas Halim & Adrienne M. Lucas & Umar Saif, 2022. "Engaging Teachers with Technology Increased Achievement, Bypassing Teachers Did Not," American Economic Journal: Economic Policy, American Economic Association, vol. 14(2), pages 61-90, May.
    6. Karthik Muralidharan & Paul Niehaus, 2017. "Experimentation at Scale," Journal of Economic Perspectives, American Economic Association, vol. 31(4), pages 103-124, Fall.
    7. Esposito Acosta,Bruno Nicola & Sautmann,Anja, 2022. "Adaptive Experiments for Policy Choice : Phone Calls for Home Reading in Kenya," Policy Research Working Paper Series 10098, The World Bank.
    8. Crawfurd, Lee & Evans, David K. & Hares, Susannah & Sandefur, Justin, 2023. "Live tutoring calls did not improve learning during the COVID-19 pandemic in Sierra Leone," Journal of Development Economics, Elsevier, vol. 164(C).
    9. Jörg Peters & Jörg Langbein & Gareth Roberts, 2018. "Generalization in the Tropics – Development Policy, Randomized Controlled Trials, and External Validity," The World Bank Research Observer, World Bank, vol. 33(1), pages 34-64.
    10. Maruyama, Takao & Kurosaki, Takashi, 2021. "Do remedial activities using math workbooks improve student learning? Empirical evidence from scaled-up interventions in Niger," World Development, Elsevier, vol. 148(C).
    11. Rosa Sanchis-Guarner & José Montalbán & Felix Weinhardt, 2021. "Home Broadband and Human Capital Formation," CESifo Working Paper Series 8846, CESifo.
    12. Islam, Asad & Lee, Wang-Sheng & Nicholas, Aaron, 2021. "The Effects of Chess Instruction on Academic and Non-cognitive Outcomes: Field Experimental Evidence from a Developing Country," Journal of Development Economics, Elsevier, vol. 150(C).
    13. Guido Neidhöfer & Nora Lustig & Mariano Tommasi, 2021. "Intergenerational transmission of lockdown consequences: prognosis of the longer-run persistence of COVID-19 in Latin America," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 19(3), pages 571-598, September.
    14. Werner, Katharina & Woessmann, Ludger, 2021. "The Legacy of COVID-19 in Education," IZA Discussion Papers 14796, Institute of Labor Economics (IZA).
    15. Eduard Marinov, 2019. "The 2019 Nobel Prize in Economics," Economic Thought journal, Bulgarian Academy of Sciences - Economic Research Institute, issue 6, pages 78-116.
    16. Johnston, Jamie & Ksoll, Christopher, 2022. "Effectiveness of interactive satellite-transmitted instruction: Experimental evidence from Ghanaian primary schools," Economics of Education Review, Elsevier, vol. 91(C).
    17. Andersen, Simon Calmar & Guul, Thorbjørn Sejr & Humlum, Maria Knoth, 2019. "Instruction Time, Information, and Student Achievement: Evidence from a Field Experiment," IZA Discussion Papers 12543, Institute of Labor Economics (IZA).
    18. Jere R. Behrman & C. Simon Fan & Naijia Guo & Xiangdong Wei & Hongliang Zhang & Junsen Zhang, 2024. "Tutoring Efficacy, Household Substitution, And Student Achievement: Experimental Evidence From An After‐School Tutoring Program In Rural China," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 65(1), pages 149-189, February.
    19. Islam, Asad, 2019. "Parent–teacher meetings and student outcomes: Evidence from a developing country," European Economic Review, Elsevier, vol. 111(C), pages 273-304.
    20. Allen, James & Mahumane, Arlete & Riddell, James & Rosenblat, Tanya & Yang, Dean & Yu, Hang, 2022. "Teaching and incentives: Substitutes or complements?," Economics of Education Review, Elsevier, vol. 91(C).

    More about this item

    JEL classification:

    • I10 - Health, Education, and Welfare - - Health - - - General
    • I20 - Health, Education, and Welfare - - Education - - - General
    • I24 - Health, Education, and Welfare - - Education - - - Education and Inequality
    • I25 - Health, Education, and Welfare - - Education - - - Education and Economic Development

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:28205. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.