IDEAS home Printed from https://ideas.repec.org/a/nat/nathum/v7y2023i3d10.1038_s41562-022-01506-4.html
   My bibliography  Save this article

A systematic review and meta-analysis of the evidence on learning during the COVID-19 pandemic

Author

Listed:
  • Bastian A. Betthäuser

    (Centre for Research on Social Inequalities (CRIS), Sciences Po
    University of Oxford
    University of Oxford)

  • Anders M. Bach-Mortensen

    (University of Oxford)

  • Per Engzell

    (University of Oxford
    University College London
    Stockholm University)

Abstract

To what extent has the learning progress of school-aged children slowed down during the COVID-19 pandemic? A growing number of studies address this question, but findings vary depending on context. Here we conduct a pre-registered systematic review, quality appraisal and meta-analysis of 42 studies across 15 countries to assess the magnitude of learning deficits during the pandemic. We find a substantial overall learning deficit (Cohen’s d = −0.14, 95% confidence interval −0.17 to −0.10), which arose early in the pandemic and persists over time. Learning deficits are particularly large among children from low socio-economic backgrounds. They are also larger in maths than in reading and in middle-income countries relative to high-income countries. There is a lack of evidence on learning progress during the pandemic in low-income countries. Future research should address this evidence gap and avoid the common risks of bias that we identify.

Suggested Citation

  • Bastian A. Betthäuser & Anders M. Bach-Mortensen & Per Engzell, 2023. "A systematic review and meta-analysis of the evidence on learning during the COVID-19 pandemic," Nature Human Behaviour, Nature, vol. 7(3), pages 375-385, March.
  • Handle: RePEc:nat:nathum:v:7:y:2023:i:3:d:10.1038_s41562-022-01506-4
    DOI: 10.1038/s41562-022-01506-4
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41562-022-01506-4
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1038/s41562-022-01506-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Aucejo, Esteban M. & Romano, Teresa Foy, 2016. "Assessing the effect of school days and absences on test score performance," Economics of Education Review, Elsevier, vol. 55(C), pages 70-87.
    2. Contini Dalit & Di Tommaso Maria Laura & Muratori Caterina & Piazzalunga Daniela & Schiavon Lucia, 2022. "Who Lost the Most? Mathematics Achievement during the COVID-19 Pandemic," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 22(2), pages 399-408, April.
    3. Noam Angrist & Simeon Djankov & Pinelopi K. Goldberg & Harry A. Patrinos, 2021. "Measuring human capital using global learning data," Nature, Nature, vol. 592(7854), pages 403-408, April.
    4. Joshua Goodman, 2014. "Flaking Out: Student Absences and Snow Days as Disruptions of Instructional Time," NBER Working Papers 20221, National Bureau of Economic Research, Inc.
    5. Nicola Fuchs-Schünde & Dirk Krueger & Alexander Ludwig & Irina Popova, 2022. "The Long-Term Distributional and Welfare Effects of Covid-19 School Closures," The Economic Journal, Royal Economic Society, vol. 132(645), pages 1647-1683.
    6. David Jaume & Alexander Willén, 2019. "The Long-Run Effects of Teacher Strikes: Evidence from Argentina," Journal of Labor Economics, University of Chicago Press, vol. 37(4), pages 1097-1139.
    7. Andrea Ichino & Rudolf Winter-Ebmer, 2004. "The Long-Run Educational Cost of World War II," Journal of Labor Economics, University of Chicago Press, vol. 22(1), pages 57-86, January.
    8. Michèle Belot & Dinand Webbink, 2010. "Do Teacher Strikes Harm Educational Attainment of Students?," LABOUR, CEIS, vol. 24(4), pages 391-406, December.
    9. Noam Angrist & Peter Bergman & Moitshepi Matsheng, 2022. "Experimental evidence on learning using low-tech when school is out," Nature Human Behaviour, Nature, vol. 6(7), pages 941-950, July.
    10. Lee Elliot Major & Andrew Eyles & Stephen Machin, 2021. "Learning loss since lockdown: variation across the home nations," CEP Covid-19 Analyses cepcovid-19-023, Centre for Economic Performance, LSE.
    11. Lee Elliot Major & Andrew Eyles & Stephen Machin, 2021. "Learning loss since lockdown," CentrePiece - The magazine for economic performance 614, Centre for Economic Performance, LSE.
    12. Kaffenberger, Michelle, 2021. "Modelling the long-run learning impact of the Covid-19 learning shock: Actions to (more than) mitigate loss," International Journal of Educational Development, Elsevier, vol. 81(C).
    13. Hevia, Felipe J. & Vergara-Lope, Samana & Velásquez-Durán, Anabel & Calderón, David, 2022. "Estimation of the fundamental learning loss and learning poverty related to COVID-19 pandemic in Mexico," International Journal of Educational Development, Elsevier, vol. 88(C).
    14. Ardington, Cally & Wills, Gabrielle & Kotze, Janeli, 2021. "COVID-19 learning losses: Early grade reading in South Africa," International Journal of Educational Development, Elsevier, vol. 86(C).
    15. Nicola Bazoli & Sonia Marzadro & Antonio Schizzerotto & Loris Vergolini, 2022. "Learning Loss and Students’ Social Origins During the Covid-19 Pandemic in Italy," FBK-IRVAPP Working Papers 2022-03, Research Institute for the Evaluation of Public Policies (IRVAPP), Bruno Kessler Foundation.
    16. Per Engzell & Arun Frey & Mark D. Verhagen, 2021. "Learning loss due to school closures during the COVID-19 pandemic," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 118(17), pages 2022376118-, April.
    17. Guilherme Lichand & Carlos Alberto Doria & Onicio Leal-Neto & João Paulo Cossi Fernandes, 2022. "Publisher Correction: The impacts of remote learning in secondary education during the pandemic in Brazil," Nature Human Behaviour, Nature, vol. 6(8), pages 1180-1180, August.
    18. João Pedro & Amer Hasan & Diana Goldemberg & Koen Geven & Syedah Aroob Iqbal, 2021. "Simulating the Potential Impacts of COVID-19 School Closures on Schooling and Learning Outcomes: A Set of Global Estimates [Tackling Inequity in Education during and after COVID-19]," The World Bank Research Observer, World Bank, vol. 36(1), pages 1-40.
    19. Patrinos,Harry Anthony & Vegas,Emiliana & Carter-Rau,Rohan, 2022. "An Analysis of COVID-19 Student Learning Loss," Policy Research Working Paper Series 10033, The World Bank.
    20. Angrist, Noam & de Barros, Andreas & Bhula, Radhika & Chakera, Shiraz & Cummiskey, Chris & DeStefano, Joseph & Floretta, John & Kaffenberger, Michelle & Piper, Benjamin & Stern, Jonathan, 2021. "Building back better to avert a learning catastrophe: Estimating learning loss from COVID-19 school shutdowns in Africa and facilitating short-term and long-term learning recovery," International Journal of Educational Development, Elsevier, vol. 84(C).
    21. Guilherme Lichand & Carlos Alberto Doria & Onicio Leal-Neto & João Paulo Cossi Fernandes, 2022. "The impacts of remote learning in secondary education during the pandemic in Brazil," Nature Human Behaviour, Nature, vol. 6(8), pages 1079-1086, August.
    22. Sébastien Goudeau & Camille Sanrey & Arnaud Stanczak & Antony Manstead & Céline Darnon, 2021. "Why lockdown and distance learning during the COVID-19 pandemic are likely to increase the social class achievement gap," Nature Human Behaviour, Nature, vol. 5(10), pages 1273-1281, October.
    23. Andreu Arenas & Lucas Gortazar, 2022. "Learning loss one year after school closures: Evidence from the Basque Country," Working Papers 2022/03, Institut d'Economia de Barcelona (IEB).
    24. Dalton Conley & Tim Johnson, 2021. "Opinion: Past is future for the era of COVID-19 research in the social sciences," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 118(13), pages 2104155118-, March.
    25. Zachary Parolin & Emma K. Lee, 2021. "Large socio-economic, geographic and demographic disparities exist in exposure to school closures," Nature Human Behaviour, Nature, vol. 5(4), pages 522-528, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kóczán, Zs., 2024. "Lasting scars: The long-term effects of school closures on earnings," World Development, Elsevier, vol. 176(C).
    2. Gajderowicz, Tomasz & Jakubowski, Maciej & Kennedy, Alec & Christrup Kjeldsen, Christian & Patrinos, Harry Anthony & Strietholt, Rolf, 2024. "The Learning Crisis: Three Years After Covid-19," GLO Discussion Paper Series 1543, Global Labor Organization (GLO).
    3. ASAKAWA Shinsuke & OHTAKE Fumio & SANO Shinpei, 2023. "The Impact of the COVID-19 Pandemic on the Academic Achievement of Elementary and Junior High School Students: Analysis using administrative data from Amagasaki City," Discussion papers 23066, Research Institute of Economy, Trade and Industry (RIETI).
    4. Ardington, Cally & Wills, Gabrielle & Kotze, Janeli, 2021. "COVID-19 learning losses: Early grade reading in South Africa," International Journal of Educational Development, Elsevier, vol. 86(C).
    5. Székely, Miguel & Flores-Ceceña, Iván & Hevia, Felipe & Calderón, David, 2024. "Measuring learning losses from delayed return to school: Evidence from Mexico," International Journal of Educational Development, Elsevier, vol. 106(C).
    6. Kuzmanic, Danilo & Valenzuela, Juan Pablo & Claro, Susana & Canales, Andrea & Cerda, Daniela & Undurraga, Eduardo A., 2023. "Socioeconomic disparities in the reopening of schools during the pandemic in Chile," International Journal of Educational Development, Elsevier, vol. 100(C).
    7. Monroy-Gómez-Franco, Luis & Vélez-Grajales, Roberto & López-Calva, Luis F., 2022. "The potential effects of the COVID-19 pandemic on learnings," International Journal of Educational Development, Elsevier, vol. 91(C).
    8. Mustafa Uğraş & Erdal Zengin & Stamatis Papadakis & Michail Kalogiannakis, 2023. "Early Childhood Learning Losses during COVID-19: Systematic Review," Sustainability, MDPI, vol. 15(7), pages 1-29, April.
    9. Alan, Sule & Turkum, Betul, 2024. "Abstract reasoning, theory of mind and character development in the school," Journal of Economic Behavior & Organization, Elsevier, vol. 221(C), pages 307-326.
    10. Neranjan Senarath & Wathsala Dedunupitiya & Gayanthi Ilangarathna & Isuru Thilakasiri & Bhagya Samarakoon & Lakshitha Ramanayake & Roshan Godaliyadda & Parakrama Ekanayake & Vijitha Herath & Mallika P, 2024. "Family income does not necessarily lead to a holistic e-learning experience during COVID-19: a study in Sri Lanka," Palgrave Communications, Palgrave Macmillan, vol. 11(1), pages 1-14, December.
    11. Guido Neidhöfer & Nora Lustig & Mariano Tommasi, 2021. "Intergenerational transmission of lockdown consequences: prognosis of the longer-run persistence of COVID-19 in Latin America," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 19(3), pages 571-598, September.
    12. Jo Blanden & Matthias Doepke & Jan Stuhler, 2022. "Education inequality," CEP Discussion Papers dp1849, Centre for Economic Performance, LSE.
    13. Shinsuke Asakawa & Fumio Ohtake, 2021. "Impact of Temporary School Closure Due to COVID-19 on the Academic Achievement of Elementary School Students," Discussion Papers in Economics and Business 21-14, Osaka University, Graduate School of Economics.
    14. OIKAWA Masato & TANAKA Ryuichi & BESSHO Shun-ichiro & KAWAMURA Akira & NOGUCHI Haruko, 2022. "Do Class Closures Affect Students' Achievements? Heterogeneous effects of students' socioeconomic backgrounds," Discussion papers 22042, Research Institute of Economy, Trade and Industry (RIETI).
    15. Neidhöfer, Guido & Lustig, Nora & Larroulet, Patricio, 2022. "Nowcasting the impact of COVID-19 on education, intergenerational mobility and earnings inequality in Sub-Saharan Africa," ZEW Discussion Papers 22-022, ZEW - Leibniz Centre for European Economic Research.
    16. Marín Llanes, Lucas & Rodríguez Pico, Mariana & Maldonado, Darío & García, Sandra, 2023. "Learning inequality during Covid-19: Evidence from secondary schools in Colombia," International Journal of Educational Development, Elsevier, vol. 100(C).
    17. Valenzuela, Juan Pablo & Kuzmanic, Danilo & Cortés, Fernanda, 2024. "Socioeconomic inequalities in opportunities and participation in in-person learning during the Covid-19 pandemic," International Journal of Educational Development, Elsevier, vol. 105(C).
    18. Noam Angrist & Peter Bergman & Moitshepi Matsheng, 2022. "Experimental evidence on learning using low-tech when school is out," Nature Human Behaviour, Nature, vol. 6(7), pages 941-950, July.
    19. Nicola Fuchs-Schündeln & Dirk Krueger & André Kurmann & Etienne Lalé & Alexander Ludwig & Irina Popova, 2023. "The Fiscal and Welfare Effects of Policy Responses to the Covid-19 School Closures," IMF Economic Review, Palgrave Macmillan;International Monetary Fund, vol. 71(1), pages 35-98, March.
    20. Emily Beam & Priya Mukherjee & Laia Navarro-Sola, 2022. "Lowering Barriers to Remote Education: Experimental Impacts on Parental Responses and Learning," Working Papers 2022-030, Human Capital and Economic Opportunity Working Group.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:nathum:v:7:y:2023:i:3:d:10.1038_s41562-022-01506-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.