IDEAS home Printed from https://ideas.repec.org/p/osf/socarx/ve4z7_v1.html
   My bibliography  Save this paper

Learning Loss Due to School Closures During the COVID-19 Pandemic

Author

Listed:
  • Engzell, Per
  • Frey, Arun
  • Verhagen, Mark D.

Abstract

Suspension of face-to-face instruction in schools during the COVID-19 pandemic has led to concerns about consequences for student learning. So far, data to study this question have been limited. Here we evaluate the effect of school closures on primary school performance using exceptionally rich data from the Netherlands (n≈350,000). We use the fact that national exams took place before and after lockdown, and compare progress during this period to the same period in the three previous years. The Netherlands underwent only a relatively short lockdown (8 weeks), and features an equitable system of school funding and the world's highest rate of broadband access. Still, our results reveal a learning loss of about 3 percentile points or 0.08 standard deviations. The effect is equivalent to a fifth of a school year, the same period that schools remained closed. Losses are up to 60% larger among students from less-educated homes, confirming worries about the uneven toll of the pandemic on children and families. Investigating mechanisms, we find that most of the effect reflects the cumulative impact of knowledge learned rather than transitory influences on the day of testing. Results remain robust when balancing on the estimated propensity of treatment and using maximum entropy weights, or with fixed-effects specifications that compare students within the same school and family. The findings imply that students made little or no progress whilst learning from home, and suggest losses even larger in countries with weaker infrastructure or longer school closures.

Suggested Citation

  • Engzell, Per & Frey, Arun & Verhagen, Mark D., 2020. "Learning Loss Due to School Closures During the COVID-19 Pandemic," SocArXiv ve4z7_v1, Center for Open Science.
  • Handle: RePEc:osf:socarx:ve4z7_v1
    DOI: 10.31219/osf.io/ve4z7_v1
    as

    Download full text from publisher

    File URL: https://osf.io/download/5f995b4687b7df03233b06fe/
    Download Restriction: no

    File URL: https://libkey.io/10.31219/osf.io/ve4z7_v1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Aigner, Dennis J., 1973. "Regression with a binary independent variable subject to errors of observation," Journal of Econometrics, Elsevier, vol. 1(1), pages 49-59, March.
    2. Hainmueller, Jens, 2012. "Entropy Balancing for Causal Effects: A Multivariate Reweighting Method to Produce Balanced Samples in Observational Studies," Political Analysis, Cambridge University Press, vol. 20(1), pages 25-46, January.
    3. John M. Abowd & Martha H. Stinson, 2013. "Estimating Measurement Error in Annual Job Earnings: A Comparison of Survey and Administrative Data," The Review of Economics and Statistics, MIT Press, vol. 95(5), pages 1451-1467, December.
    4. Nicola Fuchs-Schünde & Dirk Krueger & Alexander Ludwig & Irina Popova, 2022. "The Long-Term Distributional and Welfare Effects of Covid-19 School Closures," The Economic Journal, Royal Economic Society, vol. 132(645), pages 1647-1683.
    5. Baron, E. Jason & Goldstein, Ezra G. & Wallace, Cullen T., 2020. "Suffering in silence: How COVID-19 school closures inhibit the reporting of child maltreatment," Journal of Public Economics, Elsevier, vol. 190(C).
    6. John Jerrim & John Micklewright, 2012. "Socioeconomic gradients in children's cognitive skills: Are cross-country comparisons robust to who reports family background?," DoQSS Working Papers 12-06, Quantitative Social Science - UCL Social Research Institute, University College London.
    7. David Jaume & Alexander Willén, 2019. "The Long-Run Effects of Teacher Strikes: Evidence from Argentina," Journal of Labor Economics, University of Chicago Press, vol. 37(4), pages 1097-1139.
    8. Bruce Sacerdote, 2012. "When the Saints Go Marching Out: Long-Term Outcomes for Student Evacuees from Hurricanes Katrina and Rita," American Economic Journal: Applied Economics, American Economic Association, vol. 4(1), pages 109-135, January.
    9. Adams-Prassl, Abi & Boneva, Teodora & Golin, Marta & Rauh, Christopher, 2020. "Inequality in the impact of the coronavirus shock: Evidence from real time surveys," Journal of Public Economics, Elsevier, vol. 189(C).
    10. Jeffrey M Wooldridge, 2010. "Econometric Analysis of Cross Section and Panel Data," MIT Press Books, The MIT Press, edition 2, volume 1, number 0262232588, December.
    11. Ritzen, Jozef M. M. & van Dommelen, Jan & De Vijlder, Frans J., 1997. "School finance and school choice in the Netherlands," Economics of Education Review, Elsevier, vol. 16(3), pages 329-335, June.
    12. Victor Lavy, 2015. "Do Differences in Schools' Instruction Time Explain International Achievement Gaps? Evidence from Developed and Developing Countries," Economic Journal, Royal Economic Society, vol. 125(588), pages 397-424, November.
    13. Andrea Ichino & Rudolf Winter-Ebmer, 2004. "The Long-Run Educational Cost of World War II," Journal of Labor Economics, University of Chicago Press, vol. 22(1), pages 57-86, January.
    14. Adams-Prassl, A. & Boneva, T. & Golin, M & Rauh, C., 2020. "Inequality in the Impact of the Coronavirus Shock: New Survey Evidence for the UK," Cambridge Working Papers in Economics 2023, Faculty of Economics, University of Cambridge.
    15. Borghans Lex & Golsteyn Bart H. H. & Zölitz Ulf, 2015. "Parental Preferences for Primary School Characteristics," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 15(1), pages 85-117, January.
    16. Michèle Belot & Dinand Webbink, 2010. "Do Teacher Strikes Harm Educational Attainment of Students?," LABOUR, CEIS, vol. 24(4), pages 391-406, December.
    17. Bollinger, Christopher R., 1996. "Bounding mean regressions when a binary regressor is mismeasured," Journal of Econometrics, Elsevier, vol. 73(2), pages 387-399, August.
    18. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    19. Bol, Thijs, 2020. "Inequality in homeschooling during the Corona crisis in the Netherlands. First results from the LISS Panel," SocArXiv hf32q, Center for Open Science.
    20. Magnus Carlsson & Gordon B. Dahl & Björn Öckert & Dan-Olof Rooth, 2015. "The Effect of Schooling on Cognitive Skills," The Review of Economics and Statistics, MIT Press, vol. 97(3), pages 533-547, July.
    21. Dave E. Marcotte & Steven W. Hemelt, 2008. "Unscheduled School Closings and Student Performance," Education Finance and Policy, MIT Press, vol. 3(3), pages 316-338, July.
    22. Helen F. Ladd & Edward B. Fiske, 2011. "Weighted student funding in the Netherlands: A model for the U.S.?," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 30(3), pages 470-498, June.
    23. Bacher-Hicks, Andrew & Goodman, Joshua & Mulhern, Christine, 2021. "Inequality in household adaptation to schooling shocks: Covid-induced online learning engagement in real time," Journal of Public Economics, Elsevier, vol. 193(C).
    24. AIGNER, Dennis J., 1973. "Regression with a binary independent variable subject to errors of observation," LIDAM Reprints CORE 130, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    25. Uri Simonsohn & Joseph P. Simmons & Leif D. Nelson, 2020. "Specification curve analysis," Nature Human Behaviour, Nature, vol. 4(11), pages 1208-1214, November.
    26. Arie Kapteyn & Jelmer Y. Ypma, 2007. "Measurement Error and Misclassification: A Comparison of Survey and Administrative Data," Journal of Labor Economics, University of Chicago Press, vol. 25(3), pages 513-551.
    27. Black, Dan & Sanders, Seth & Taylor, Lowell, 2003. "Measurement of Higher Education in the Census and Current Population Survey," Journal of the American Statistical Association, American Statistical Association, vol. 98, pages 545-554, January.
    28. John Jerrim & Anna Vignoles, 2013. "Social mobility, regression to the mean and the cognitive development of high ability children from disadvantaged homes," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 176(4), pages 887-906, October.
    29. Nienke Ruijs & Hessel Oosterbeek, 2019. "School Choice in Amsterdam: Which Schools are Chosen When School Choice is Free?," Education Finance and Policy, MIT Press, vol. 14(1), pages 1-30, Winter.
    30. Uri Simonsohn & Joseph P. Simmons & Leif D. Nelson, 2020. "Publisher Correction: Specification curve analysis," Nature Human Behaviour, Nature, vol. 4(11), pages 1215-1215, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kóczán, Zs., 2024. "Lasting scars: The long-term effects of school closures on earnings," World Development, Elsevier, vol. 176(C).
    2. Guido Neidhöfer & Nora Lustig & Mariano Tommasi, 2021. "Intergenerational transmission of lockdown consequences: prognosis of the longer-run persistence of COVID-19 in Latin America," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 19(3), pages 571-598, September.
    3. Kamila Cygan-Rehm, 2022. "Lifetime Consequences of Lost Instructional Time in the Classroom: Evidence from Shortened School Years," CESifo Working Paper Series 9892, CESifo.
    4. Engzell, Per & Frey, Arun & Verhagen, Mark D., 2020. "Learning Inequality During the Covid-19 Pandemic," SocArXiv ve4z7, Center for Open Science.
    5. Nicola Fuchs-Schünde & Dirk Krueger & Alexander Ludwig & Irina Popova, 2022. "The Long-Term Distributional and Welfare Effects of Covid-19 School Closures," The Economic Journal, Royal Economic Society, vol. 132(645), pages 1647-1683.
    6. Nicola Fuchs-Schündeln & Dirk Krueger & André Kurmann & Etienne Lalé & Alexander Ludwig & Irina Popova, 2023. "The Fiscal and Welfare Effects of Policy Responses to the Covid-19 School Closures," IMF Economic Review, Palgrave Macmillan;International Monetary Fund, vol. 71(1), pages 35-98, March.
    7. Monroy-Gómez-Franco, Luis & Vélez-Grajales, Roberto & López-Calva, Luis F., 2022. "The potential effects of the COVID-19 pandemic on learnings," International Journal of Educational Development, Elsevier, vol. 91(C).
    8. Jo Blanden & Matthias Doepke & Jan Stuhler, 2022. "Education inequality," CEP Discussion Papers dp1849, Centre for Economic Performance, LSE.
    9. Monroy-Gómez-Franco, Luis Angel, 2021. "Modelling the learning impacts of educational disruptions in the short and long run," SocArXiv z6x5s, Center for Open Science.
    10. Takahide Yanagi, 2019. "Inference on local average treatment effects for misclassified treatment," Econometric Reviews, Taylor & Francis Journals, vol. 38(8), pages 938-960, September.
    11. Oikawa, Masato & Tanaka, Ryuichi & Bessho, Shun-ichiro & Kawamura, Akira & Noguchi, Haruko, 2022. "Do Class Closures Affect Students' Achievements? Heterogeneous Effects of Students' Socioeconomic Backgrounds," IZA Discussion Papers 15751, Institute of Labor Economics (IZA).
    12. Giorgio Di Pietro & Federico Biagi & Patricia Costa & Zbigniew Karpinski & Jacopo Mazza, 2020. "The likely impact of COVID-19 on education: Reflections based on the existing literature and recent international datasets," JRC Research Reports JRC121071, Joint Research Centre.
    13. Shinsuke Asakawa & Fumio Ohtake, 2021. "Impact of Temporary School Closure Due to COVID-19 on the Academic Achievement of Elementary School Students," Discussion Papers in Economics and Business 21-14, Osaka University, Graduate School of Economics.
    14. Craig Gundersen & Brent Kreider, 2008. "Food Stamps and Food Insecurity: What Can Be Learned in the Presence of Nonclassical Measurement Error?," Journal of Human Resources, University of Wisconsin Press, vol. 43(2), pages 352-382.
    15. Erich Battistin & Barbara Sianesi, 2006. "Misreported schooling and returns to education: evidence from the UK," CeMMAP working papers CWP07/06, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    16. Mandila Ben Shikomera & Collins K. Matemba & Joash Migosi, 2024. "Emerging Issues in Curriculum and Instruction during Post COVID-19 Era: Experience from South Eastern Kenya University, Kenya," International Journal of Research and Innovation in Social Science, International Journal of Research and Innovation in Social Science (IJRISS), vol. 8(4), pages 1743-1763, April.
    17. Atolia, Manoj & Papageorgiou, Chris & Turnovsky, Stephen J., 2021. "Re-opening after the lockdown: Long-run aggregate and distributional consequences of COVID-19," Journal of Mathematical Economics, Elsevier, vol. 93(C).
    18. Youngsoo Jang & Minchul Yum, 2024. "Aggregate and Intergenerational Implications of School Closures: A Quantitative Assessment," American Economic Journal: Macroeconomics, American Economic Association, vol. 16(3), pages 90-130, July.
    19. Werner, Katharina & Woessmann, Ludger, 2021. "The Legacy of COVID-19 in Education," IZA Discussion Papers 14796, Institute of Labor Economics (IZA).
    20. Schüller, Simone & Steinberg, Hannah S., 2022. "Parents under stress: Evaluating emergency childcare policies during the first COVID-19 lockdown in Germany," Labour Economics, Elsevier, vol. 78(C).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:socarx:ve4z7_v1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://arabixiv.org .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.