IDEAS home Printed from https://ideas.repec.org/p/qss/dqsswp/2112.html
   My bibliography  Save this paper

A review and evaluation of secondary school accountability in England: Statistical strengths, weaknesses, and challenges for ‘Progress 8’ raised by COVID-19

Author

Listed:
  • Lucy Prior

    (Centre for Multilevel Modelling, School of Education, University of Bristol, UK)

  • John Jerrim

    (Department of Quantitative Science, UCL Social Research Institute, IOE, UK)

  • Dave Thomson

    (FFT Education Datalab, London, UK)

  • George Leckie

    (Centre for Multilevel Modelling, School of Education, University of Bristol, UK)

Abstract

School performance measures are published annually in England to hold schools to account and to support parental school choice. This article reviews and evaluates the ‘Progress 8’ secondary school accountability system for state-funded schools. We assess the statistical strengths and weaknesses of Progress 8 relating to: choice of pupil outcome attainment measure; potential adjustments for pupil input attainment and background characteristics; decisions around which schools and pupils are excluded from the measure; presentation of Progress 8 to users, choice of statistical model, and calculation of statistical uncertainty; and issues related to the volatility of school performance over time, including scope for reporting multi-year averages. We then discuss challenges for Progress 8 raised by the COVID-19 pandemic. Six simple recommendations follow to improve Progress 8 and school accountability in England.

Suggested Citation

  • Lucy Prior & John Jerrim & Dave Thomson & George Leckie, 2021. "A review and evaluation of secondary school accountability in England: Statistical strengths, weaknesses, and challenges for ‘Progress 8’ raised by COVID-19," DoQSS Working Papers 21-12, Quantitative Social Science - UCL Social Research Institute, University College London.
    • Handle: RePEc:qss:dqsswp:2112
    as

    Download full text from publisher

    File URL: http://repec.ioe.ac.uk/REPEc/pdf/qsswp2112.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. George Leckie & Harvey Goldstein, 2011. "Understanding Uncertainty in School League Tables," Fiscal Studies, Institute for Fiscal Studies, vol. 32(2), pages 207-224, June.
    2. Burgess, Simon & Propper, Carol & Wilson, Deborah & Slater, Helen, 2005. "Who wins and who loses from school accountability? The distribution of educational gain in English secondary schools," CEPR Discussion Papers 5248, C.E.P.R. Discussion Papers.
    3. George Leckie, 2009. "The complexity of school and neighbourhood effects and movements of pupils on school differences in models of educational achievement," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 172(3), pages 537-554, June.
    4. John Jerrim, 2020. "Measuring socio-economic background using administrative data. What is the best proxy available?," DoQSS Working Papers 20-09, Quantitative Social Science - UCL Social Research Institute, University College London.
    5. Harvey Goldstein & Michael J. R. Healy, 1995. "The Graphical Presentation of a Collection of Means," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 158(1), pages 175-177, January.
    6. Lorraine Dearden & Alfonso Miranda & Sophia Rabe‐Hesketh, 2011. "Measuring School Value Added with Administrative Data: The Problem of Missing Variables," Fiscal Studies, Institute for Fiscal Studies, vol. 32(2), pages 263-278, June.
    7. Rebecca Allen & Simon Burgess, 2011. "Can School League Tables Help Parents Choose Schools?," Fiscal Studies, Institute for Fiscal Studies, vol. 32(2), pages 245-261, June.
    8. J Taylor, 2011. "The English Baccalaureate: how not to measure school performance," Working Papers 616395, Lancaster University Management School, Economics Department.
    9. Deborah Wilson, 2004. "Which Ranking? The Impact of a 'Value-Added' Measure of Secondary School Performance," Public Money & Management, Taylor & Francis Journals, vol. 24(1), pages 37-45.
    10. Katherine E. Castellano & Sophia Rabe-Hesketh & Anders Skrondal, 2014. "Composition, Context, and Endogeneity in School and Teacher Comparisons," Journal of Educational and Behavioral Statistics, , vol. 39(5), pages 333-367, October.
    11. George Leckie & Harvey Goldstein, 2009. "The limitations of using school league tables to inform school choice," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 172(4), pages 835-851, October.
    12. Andrew Ray & Tanya McCormack & Helen Evans, 2009. "Value Added in English Schools," Education Finance and Policy, MIT Press, vol. 4(4), pages 415-438, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lucy Prior & John Jerrim & Dave Thomson & George Leckie, 2021. "A review and evaluation of secondary school accountability in England: Statistical strengths, weaknesses, and challenges for 'Progress 8' raised by COVID-19," CEPEO Working Paper Series 21-04, UCL Centre for Education Policy and Equalising Opportunities, revised Apr 2021.
    2. Isabella Sulis & Mariano Porcu, 2015. "Assessing Divergences in Mathematics and Reading Achievement in Italian Primary Schools: A Proposal of Adjusted Indicators of School Effectiveness," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 122(2), pages 607-634, June.
    3. Juan Merlo & Philippe Wagner & Nermin Ghith & George Leckie, 2016. "An Original Stepwise Multilevel Logistic Regression Analysis of Discriminatory Accuracy: The Case of Neighbourhoods and Health," PLOS ONE, Public Library of Science, vol. 11(4), pages 1-31, April.
    4. Deborah Wilson & Anete Piebalga, 2008. "Accurate performance measure but meaningless ranking exercise? An analysis of the English school league tables," The Centre for Market and Public Organisation 07/176, The Centre for Market and Public Organisation, University of Bristol, UK.
    5. Gabriele B. Durrant & Sylke V. Schnepf, 2018. "Which schools and pupils respond to educational achievement surveys?: a focus on the English Programme for International Student Assessment sample," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 181(4), pages 1057-1075, October.
    6. Allen, Rebecca & Burgess, Simon, 2013. "Evaluating the provision of school performance information for school choice," Economics of Education Review, Elsevier, vol. 34(C), pages 175-190.
    7. Isabella Sulis & Mariano Porcu & Vincenza Capursi, 2019. "On the Use of Student Evaluation of Teaching: A Longitudinal Analysis Combining Measurement Issues and Implications of the Exercise," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 142(3), pages 1305-1331, April.
    8. Schnepf, Sylke V. & Durrant, Gabriele B. & Micklewright, John, 2014. "Which Schools and Pupils Respond to Educational Achievement Surveys? A Focus on the English PISA Sample," IZA Discussion Papers 8411, Institute of Labor Economics (IZA).
    9. Brendan Houng & Moshe Justman, 2013. "Comparing Least-Squares Value-Added Analysis and Student Growth Percentile Analysis for Evaluating Student Progress and Estimating School Effects," Melbourne Institute Working Paper Series wp2013n07, Melbourne Institute of Applied Economic and Social Research, The University of Melbourne.
    10. Leonardo Grilli & Carla Rampichini, 2015. "Specification of random effects in multilevel models: a review," Quality & Quantity: International Journal of Methodology, Springer, vol. 49(3), pages 967-976, May.
    11. Moshe Justman & Brendan Houng, 2013. "A Comparison Of Two Methods For Estimating School Effects And Tracking Student Progress From Standardized Test Scores," Working Papers 1316, Ben-Gurion University of the Negev, Department of Economics.
    12. Jon Rasbash & George Leckie & Rebecca Pillinger & Jennifer Jenkins, 2010. "Children's educational progress: partitioning family, school and area effects," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 173(3), pages 657-682, July.
    13. George Leckie, 2022. "A celebration of Harvey Goldstein’s lifetime contributions: Memories of working with Harvey Goldstein on educational research and statistics," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 185(3), pages 758-762, July.
    14. Nicholas Tibor Longford, 2016. "Decision Theory Applied to Selecting the Winners, Ranking, and Classification," Journal of Educational and Behavioral Statistics, , vol. 41(4), pages 420-442, August.
    15. Kilchan Choi & Jinok Kim, 2019. "Latent Variable Regression Four-Level Hierarchical Model Using Multisite Multiple-Cohort Longitudinal Data," Journal of Educational and Behavioral Statistics, , vol. 44(5), pages 597-624, October.
    16. Luis Alejandro Lopez-Agudo & Oscar David Marcenaro Gutierrez, 2016. "Identifying effective teachers: The case study of Spain," Investigaciones de Economía de la Educación volume 11, in: José Manuel Cordero Ferrera & Rosa Simancas Rodríguez (ed.), Investigaciones de Economía de la Educación 11, edition 1, volume 11, chapter 18, pages 349-366, Asociación de Economía de la Educación.
    17. Maria Ferrão, 2012. "On the stability of value added indicators," Quality & Quantity: International Journal of Methodology, Springer, vol. 46(2), pages 627-637, February.
    18. Columbu, Silvia & Porcu, Mariano & Sulis, Isabella, 2021. "University choice and the attractiveness of the study area: Insights on the differences amongst degree programmes in Italy based on generalised mixed-effect models," Socio-Economic Planning Sciences, Elsevier, vol. 74(C).
    19. Garritt L. Page & Ernesto San Martín & Javiera Orellana & Jorge González, 2017. "Exploring complete school effectiveness via quantile value added," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 180(1), pages 315-340, January.
    20. Gordey A. Yasterbov & Alexey R. Bessudnov & Marina A. Pinskaya & Sergey G. Kosaretsky, 2014. "Contextualizing Academic Performance In Russian Schools: School Characteristics, The Composition Of Student Body And Local Deprivation," HSE Working papers WP BRP 55/SOC/2014, National Research University Higher School of Economics.

    More about this item

    Keywords

    Progress 8; school performance measures; school accountability; school choice; school league tables; value-added model; COVID-19;
    All these keywords.

    JEL classification:

    • I2 - Health, Education, and Welfare - - Education

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:qss:dqsswp:2112. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Dr Neus Bover Fonts (email available below). General contact details of provider: https://edirc.repec.org/data/dqioeuk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.