IDEAS home Printed from https://ideas.repec.org/p/qss/dqsswp/1105.html
   My bibliography  Save this paper

Measuring school value added with administrative data: the problem of missing variables

Author

Listed:
  • Lorraine Dearden

    (Institute for Fiscal Studies, 7 Ridgmount Street, London, WC1E 7AE; Institute of Education, University of London, 20 Bedford Way, London WC1H 0AL, UK.)

  • Alfonso Miranda

    (Department of Quantitative Social Science, Institute of Education, University of London. 20 Bedford Way, London WC1H 0AL, UK.)

  • Sophia Rabe-Hesketh

    (Graduate School of Education and Graduate Group in Biostatistics, University of California, Berkeley, USA. Institute of Education, University of London, London, UK.)

Abstract

The UK Department for Education (DfE) calculates contextualised value added (CVA) measures of school performance using administrative data that contain only a limited set of explanatory variables. Differences on schools’ intake regarding characteristics such as mother’s education are not accounted for due to the lack of background information in the data. In this paper we use linked survey and administrative data to assess the potential biases that missing control variables cause in the calculation of CVA measures of school performance. We find that ignoring the effect of mother’s education leads DfE to erroneously over-penalise low achieving schools that have a greater proportion of mothers with low qualifications and to over-reward high achieving schools that have a greater proportion of mothers with higher qualifications. This suggests that collecting a rich set of controls in administrative records is necessary for producing reliable CVA measures of school performance.

Suggested Citation

  • Lorraine Dearden & Alfonso Miranda & Sophia Rabe-Hesketh, 2011. "Measuring school value added with administrative data: the problem of missing variables," DoQSS Working Papers 11-05, Quantitative Social Science - UCL Social Research Institute, University College London.
    • Handle: RePEc:qss:dqsswp:1105
    as

    Download full text from publisher

    File URL: https://repec.ucl.ac.uk/REPEc/pdf/qsswp1105.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. George Leckie & Harvey Goldstein, 2009. "The limitations of using school league tables to inform school choice," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 172(4), pages 835-851, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Gabriele B. Durrant & Sylke V. Schnepf, 2018. "Which schools and pupils respond to educational achievement surveys?: a focus on the English Programme for International Student Assessment sample," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 181(4), pages 1057-1075, October.
    2. Lucy Prior & John Jerrim & Dave Thomson & George Leckie, 2021. "A review and evaluation of secondary school accountability in England: Statistical strengths, weaknesses, and challenges for 'Progress 8' raised by COVID-19," CEPEO Working Paper Series 21-04, UCL Centre for Education Policy and Equalising Opportunities, revised Apr 2021.
    3. Sofia N. Andreou & Panos Pashardes, 2012. "Consumers’ Valuation of Level and Egalitarian Education Attainment of Schools in England," University of Cyprus Working Papers in Economics 10-2012, University of Cyprus Department of Economics.
    4. Alfonso Miranda & Sophia Rabe-Hesketh & John W. McDonald, 2012. "Reducing bias due to missing values of the response variable by joint modeling with an auxiliary variable," DoQSS Working Papers 12-05, Quantitative Social Science - UCL Social Research Institute, University College London.
    5. Sofia N. Andreou & Panos Pashardes, 2013. "Consumers’ Valuation of Academic and Equality-inducing Aspects of School Performance in England," University of Cyprus Working Papers in Economics 09-2013, University of Cyprus Department of Economics.
    6. Schnepf, Sylke V. & Durrant, Gabriele B. & Micklewright, John, 2014. "Which Schools and Pupils Respond to Educational Achievement Surveys? A Focus on the English PISA Sample," IZA Discussion Papers 8411, Institute of Labor Economics (IZA).
    7. Doris, Aedin & O'Neill, Donal & Sweetman, Olive, 2019. "Good Schools or Good Students? The Importance of Selectivity for School Rankings," IZA Discussion Papers 12459, Institute of Labor Economics (IZA).
    8. Oketch, Moses & Rolleston, Caine & Rossiter, Jack, 2021. "Diagnosing the learning crisis: What can value-added analysis contribute?," International Journal of Educational Development, Elsevier, vol. 87(C).
    9. Lucy Prior & John Jerrim & Dave Thomson & George Leckie, 2021. "A review and evaluation of secondary school accountability in England: Statistical strengths, weaknesses, and challenges for ‘Progress 8’ raised by COVID-19," DoQSS Working Papers 21-12, Quantitative Social Science - UCL Social Research Institute, University College London.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Arpino, Bruno & Varriale, Roberta, 2009. "Assessing the quality of institutions’ rankings obtained through multilevel linear regression models," MPRA Paper 19873, University Library of Munich, Germany.
    2. Claudia Herresthal, 2015. "Inferring School Quality from Rankings: The Impact of School Choice," Economics Series Working Papers 747, University of Oxford, Department of Economics.
    3. Sulis, Isabella & Giambona, Francesca & Porcu, Mariano, 2020. "Adjusted indicators of quality and equity for monitoring the education systems over time. Insights on EU15 countries from PISA surveys," Socio-Economic Planning Sciences, Elsevier, vol. 69(C).
    4. Isabella Sulis & Mariano Porcu, 2012. "Comparing degree programs from students’ assessments: A LCRA-based adjusted composite indicator," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 21(2), pages 193-209, June.
    5. Francesca Giambona & Mariano Porcu & Isabella Sulis, 2017. "Students Mobility: Assessing the Determinants of Attractiveness Across Competing Territorial Areas," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 133(3), pages 1105-1132, September.
    6. Luque, Mariano & Marcenaro-Gutierrez, Oscar D. & Ruiz, Ana B., 2020. "Evaluating the global efficiency of teachers through a multi-criteria approach," Socio-Economic Planning Sciences, Elsevier, vol. 70(C).
    7. Allen, Rebecca & Burgess, Simon, 2013. "Evaluating the provision of school performance information for school choice," Economics of Education Review, Elsevier, vol. 34(C), pages 175-190.
    8. Lucy Prior & John Jerrim & Dave Thomson & George Leckie, 2021. "A review and evaluation of secondary school accountability in England: Statistical strengths, weaknesses, and challenges for 'Progress 8' raised by COVID-19," CEPEO Working Paper Series 21-04, UCL Centre for Education Policy and Equalising Opportunities, revised Apr 2021.
    9. Bruno ARPINO & Roberta VARRIALE, 2010. "Assessing The Quality Of Institutions’ Rankings Obtained Through Multilevel Linear Regression Models," Journal of Applied Economic Sciences, Spiru Haret University, Faculty of Financial Management and Accounting Craiova, vol. 5(1(11)_Spr), pages 7-22.
    10. Gerdes, Christer, 2015. "Does Performance Information Affect Job Seekers in Selecting Private Providers in Voucher-Based ALMP Programs?," IZA Discussion Papers 8992, Institute of Labor Economics (IZA).
    11. Rebecca Allen, 2013. "Measuring foundation school effectiveness using English administrative data, survey data and a regression discontinuity design," Education Economics, Taylor & Francis Journals, vol. 21(5), pages 431-446, December.
    12. Nicholas T. Longford, 2020. "Performance assessment as an application of causal inference," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 183(4), pages 1363-1385, October.
    13. Steele, Fiona & Clarke, Paul & Kuha, Jouni, 2019. "Modeling within-household associations in household panel studies," LSE Research Online Documents on Economics 88162, London School of Economics and Political Science, LSE Library.
    14. Lucy Prior & John Jerrim & Dave Thomson & George Leckie, 2021. "A review and evaluation of secondary school accountability in England: Statistical strengths, weaknesses, and challenges for ‘Progress 8’ raised by COVID-19," DoQSS Working Papers 21-12, Quantitative Social Science - UCL Social Research Institute, University College London.
    15. Isabella Sulis & Mariano Porcu, 2015. "Assessing Divergences in Mathematics and Reading Achievement in Italian Primary Schools: A Proposal of Adjusted Indicators of School Effectiveness," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 122(2), pages 607-634, June.
    16. Isabella Sulis & Mariano Porcu & Vincenza Capursi, 2019. "On the Use of Student Evaluation of Teaching: A Longitudinal Analysis Combining Measurement Issues and Implications of the Exercise," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 142(3), pages 1305-1331, April.
    17. Juan Merlo & Philippe Wagner & Nermin Ghith & George Leckie, 2016. "An Original Stepwise Multilevel Logistic Regression Analysis of Discriminatory Accuracy: The Case of Neighbourhoods and Health," PLOS ONE, Public Library of Science, vol. 11(4), pages 1-31, April.
    18. Roxanne Connelly & Susan J. Murray & Vernon Gayle, 2013. "Young People and School GCSE Attainment: Exploring the ‘Middle’," Sociological Research Online, , vol. 18(1), pages 210-221, February.
    19. Nicholas T. Longford, 2012. "A Revision of School Effectiveness Analysis," Journal of Educational and Behavioral Statistics, , vol. 37(1), pages 157-179, February.
    20. George Leckie & Rebecca Pillinger & Kelvyn Jones & Harvey Goldstein, 2012. "Multilevel Modeling of Social Segregation," Journal of Educational and Behavioral Statistics, , vol. 37(1), pages 3-30, February.

    More about this item

    Keywords

    contextualised value added; missing data; informative sample selection; administrative data; UK;
    All these keywords.

    JEL classification:

    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • C18 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Methodolical Issues: General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:qss:dqsswp:1105. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Dr Neus Bover Fonts (email available below). General contact details of provider: https://edirc.repec.org/data/dqioeuk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.