IDEAS home Printed from https://ideas.repec.org/a/eee/soceps/v68y2019ics0038012118302684.html
   My bibliography  Save this article

Measuring efficiency in education: The influence of imprecision and variability in data on DEA estimates

Author

Listed:
  • Aparicio, Juan
  • Cordero, Jose M.
  • Ortiz, Lidia

Abstract

Many studies devoted to efficiency performance evaluation in the education sector are based on measures of central tendency at school level as, for example, the average values of students belonging to the same school. Although this is a common and accepted way of summarizing data from the original observations (students), it is not less true that this approach neglects the existing dispersion of data, which may become a serious problem if variability across schools is high. Additionally, imprecision may arise when experts on each evaluated subject select the battery of questions, with different levels of difficulty, which will be the base for the final questionnaires completed by students. This paper uses data from US students and schools participating in PISA (Programme for International Student Assessment) 2015 to illustrate that schools' efficiency measures based on aggregate data and imprecision may reflect an inaccurate picture of their performance if they are compared to measures estimated accounting for broader information provided by all students of the same school. In order to operationalize our approach, we resort to Fuzzy Data Envelopment Analysis. This methodology allows us to deal with the notion of fuzziness in some variables such as the socio-economic status of students or test scores. Our results indicate that the estimated measures of performance obtained with the fuzzy DEA approach are highly correlated with those calculated with traditional DEA models. However, we find some relevant divergences in the identification of efficient units when we account for data dispersion and vagueness.

Suggested Citation

  • Aparicio, Juan & Cordero, Jose M. & Ortiz, Lidia, 2019. "Measuring efficiency in education: The influence of imprecision and variability in data on DEA estimates," Socio-Economic Planning Sciences, Elsevier, vol. 68(C).
    • Handle: RePEc:eee:soceps:v:68:y:2019:i:c:s0038012118302684
    DOI: 10.1016/j.seps.2019.03.004
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0038012118302684
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.seps.2019.03.004?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Leopold Simar & Paul Wilson, 2000. "A general methodology for bootstrapping in non-parametric frontier models," Journal of Applied Statistics, Taylor & Francis Journals, vol. 27(6), pages 779-802.
    2. Konstantinos Triantis & Olivier Girod, 1998. "A Mathematical Programming Approach for Measuring Technical Efficiency in a Fuzzy Environment," Journal of Productivity Analysis, Springer, vol. 10(1), pages 85-102, July.
    3. A. Bessent & W. Bessent & J. Kennington & B. Reagan, 1982. "An Application of Mathematical Programming to Assess Productivity in the Houston Independent School District," Management Science, INFORMS, vol. 28(12), pages 1355-1367, December.
    4. Léopold Simar & Paul W. Wilson, 2015. "Statistical Approaches for Non-parametric Frontier Models: A Guided Tour," International Statistical Review, International Statistical Institute, vol. 83(1), pages 77-110, April.
    5. Wilson, Paul W., 2018. "Dimension reduction in nonparametric models of production," European Journal of Operational Research, Elsevier, vol. 267(1), pages 349-367.
    6. Kristof De Witte & Laura López-Torres, 2017. "Efficiency in education: a review of literature and a way forward," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 68(4), pages 339-363, April.
    7. John Jerrim & John Micklewright, 2012. "Socioeconomic gradients in children's cognitive skills: Are cross-country comparisons robust to who reports family background?," DoQSS Working Papers 12-06, Quantitative Social Science - UCL Social Research Institute, University College London.
    8. William Cooper & Zhimin Huang & Vedran Lelas & Susan Li & Ole Olesen, 1998. "Chance Constrained Programming Formulations for Stochastic Characterizations of Efficiency and Dominance in DEA," Journal of Productivity Analysis, Springer, vol. 9(1), pages 53-79, January.
    9. Víctor Giménez & Claudio Thieme & Diego Prior & Emili Tortosa-Ausina, 2017. "An international comparison of educational systems: a temporal analysis in presence of bad outputs," Journal of Productivity Analysis, Springer, vol. 47(1), pages 83-101, February.
    10. Freeman R B. & Machin, S. J. & Viarengo, M.G, 2011. "Inequality of Educational Outcomes: International Evidence from PISA," Regional and Sectoral Economic Studies, Euro-American Association of Economic Development, vol. 11(3).
    11. TAVANA, Madjid & SHIRAZ, Rashed Khanjani & HATAMI-MARBINI, Adel & AGRELL, Per J., 2012. "Fuzzy stochastic data envelopment analysis with application to base realignment and closure (BRAC)," LIDAM Reprints CORE 2412, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    12. Cherchye, Laurens & De Witte, Kristof & Ooghe, Erwin & Nicaise, Ides, 2010. "Efficiency and equity in private and public education: A nonparametric comparison," European Journal of Operational Research, Elsevier, vol. 202(2), pages 563-573, April.
    13. Eva Crespo-Cebada & Francisco Pedraja-Chaparro & Daniel Santín, 2014. "Does school ownership matter? An unbiased efficiency comparison for regions of Spain," Journal of Productivity Analysis, Springer, vol. 41(1), pages 153-172, February.
    14. Fare, Rolf & Grosskopf, Shawna, 2004. "Modeling undesirable factors in efficiency evaluation: Comment," European Journal of Operational Research, Elsevier, vol. 157(1), pages 242-245, August.
    15. Hatami-Marbini, Adel & Emrouznejad, Ali & Tavana, Madjid, 2011. "A taxonomy and review of the fuzzy data envelopment analysis literature: Two decades in the making," European Journal of Operational Research, Elsevier, vol. 214(3), pages 457-472, November.
    16. Aparicio, Juan & Cordero, Jose M. & Pastor, Jesus T., 2017. "The determination of the least distance to the strongly efficient frontier in Data Envelopment Analysis oriented models: Modelling and computational aspects," Omega, Elsevier, vol. 71(C), pages 1-10.
    17. A. Charnes & W. W. Cooper & E. Rhodes, 1981. "Evaluating Program and Managerial Efficiency: An Application of Data Envelopment Analysis to Program Follow Through," Management Science, INFORMS, vol. 27(6), pages 668-697, June.
    18. José Manuel Cordero & Daniel Santín & Rosa Simancas, 2017. "Assessing European primary school performance through a conditional nonparametric model," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 68(4), pages 364-376, April.
    19. Robert Mislevy, 1993. "Should “multiple imputations” be treated as “multiple indicators”?," Psychometrika, Springer;The Psychometric Society, vol. 58(1), pages 79-85, March.
    20. R. D. Banker & A. Charnes & W. W. Cooper, 1984. "Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis," Management Science, INFORMS, vol. 30(9), pages 1078-1092, September.
    21. Olesen, Ole B. & Petersen, Niels Christian, 2016. "Stochastic Data Envelopment Analysis—A review," European Journal of Operational Research, Elsevier, vol. 251(1), pages 2-21.
    22. Thieme, Claudio & Prior, Diego & Tortosa-Ausina, Emili, 2013. "A multilevel decomposition of school performance using robust nonparametric frontier techniques," Economics of Education Review, Elsevier, vol. 32(C), pages 104-121.
    23. Seiford, Lawrence M. & Zhu, Joe, 2002. "Modeling undesirable factors in efficiency evaluation," European Journal of Operational Research, Elsevier, vol. 142(1), pages 16-20, October.
    24. Adel HATAMI-MARBINI & Per J. AGRELL & Madjid TAVANA & Pegah KHOSHNEVIS, 2017. "A flexible cross-efficiency fuzzy data envelopment analysis model for sustainable sourcing," LIDAM Reprints CORE 2880, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    25. Peter Bogetoft & Lars Otto, 2011. "Benchmarking with DEA, SFA, and R," International Series in Operations Research and Management Science, Springer, number 978-1-4419-7961-2, April.
    26. SAATI, Saber & HATAMI-MARBINI, Adel & TAVANA, Madjid & AGRELL, Per, 2013. "A fuzzy data envelopment analysis for clustering operating units with imprecise data," LIDAM Reprints CORE 2465, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    27. Daniel Santín & Gabriela Sicilia, 2015. "Measuring the efficiency of public schools in Uruguay: main drivers and policy implications," Latin American Economic Review, Springer;Centro de Investigaciòn y Docencia Económica (CIDE), vol. 24(1), pages 1-28, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Anna Labijak-Kowalska & Miłosz Kadziński, 2023. "Exact and stochastic methods for robustness analysis in the context of Imprecise Data Envelopment Analysis," Operational Research, Springer, vol. 23(1), pages 1-34, March.
    2. Mishra, Neelesh Kumar & Chakraborty, Abhishek & Singh, Sanjeet & Ranjan, Prabhat, 2023. "Efficiency analysis of engineering colleges in India: Decomposition into parallel sub-processes systems," Socio-Economic Planning Sciences, Elsevier, vol. 89(C).
    3. Henriques, C.O. & Chavez, J.M. & Gouveia, M.C. & Marcenaro-Gutierrez, O.D., 2022. "Efficiency of secondary schools in Ecuador: A value based DEA approach," Socio-Economic Planning Sciences, Elsevier, vol. 82(PA).
    4. Goker, Nazli & Karsak, E.Ertugrul, 2021. "Two-stage common weight DEA-Based approach for performance evaluation with imprecise data," Socio-Economic Planning Sciences, Elsevier, vol. 74(C).
    5. De la Cruz, Marco & Mergoni, Anna, 2024. "Assessing the performance of Peruvian education system from a governance perspective," Socio-Economic Planning Sciences, Elsevier, vol. 93(C).
    6. Henriques, C.O. & Marcenaro-Gutierrez, O.D., 2021. "Efficiency of secondary schools in Portugal: A novel DEA hybrid approach," Socio-Economic Planning Sciences, Elsevier, vol. 74(C).
    7. Toloo, Mehdi & Ebrahimi, Bohlool & Amin, Gholam R., 2021. "New data envelopment analysis models for classifying flexible measures: The role of non-Archimedean epsilon," European Journal of Operational Research, Elsevier, vol. 292(3), pages 1037-1050.
    8. Capasso, Salvatore & Kaisari, Maria & Kounetas, Konstantinos & Lainas, Elias, 2024. "School productive performance and technology gaps: New evidence from PISA 2018," Economic Modelling, Elsevier, vol. 131(C).
    9. Juan Aparicio & Jose M. Cordero & Lidia Ortiz, 2021. "Efficiency Analysis with Educational Data: How to Deal with Plausible Values from International Large-Scale Assessments," Mathematics, MDPI, vol. 9(13), pages 1-16, July.
    10. López-Torres, Laura & Prior, Diego, 2022. "Long-term efficiency of public service provision in a context of budget restrictions. An application to the education sector," Socio-Economic Planning Sciences, Elsevier, vol. 81(C).
    11. Rebai, Sonia & Ben Yahia, Fatma & Essid, Hédi, 2020. "A graphically based machine learning approach to predict secondary schools performance in Tunisia," Socio-Economic Planning Sciences, Elsevier, vol. 70(C).
    12. Raul Moragues & Juan Aparicio & Miriam Esteve, 2023. "Measuring technical efficiency for multi-input multi-output production processes through OneClass Support Vector Machines: a finite-sample study," Operational Research, Springer, vol. 23(3), pages 1-33, September.
    13. Jorge Antunes & Abdollah Hadi-Vencheh & Ali Jamshidi & Yong Tan & Peter Wanke, 2022. "Bank efficiency estimation in China: DEA-RENNA approach," Annals of Operations Research, Springer, vol. 315(2), pages 1373-1398, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Aparicio, Juan & Cordero, Jose M. & Gonzalez, Martin & Lopez-Espin, Jose J., 2018. "Using non-radial DEA to assess school efficiency in a cross-country perspective: An empirical analysis of OECD countries," Omega, Elsevier, vol. 79(C), pages 9-20.
    2. Cordero, Jose M. & Polo, Cristina & Santín, Daniel & Simancas, Rosa, 2018. "Efficiency measurement and cross-country differences among schools: A robust conditional nonparametric analysis," Economic Modelling, Elsevier, vol. 74(C), pages 45-60.
    3. Juan Aparicio & Jose M. Cordero & Lidia Ortiz, 2021. "Efficiency Analysis with Educational Data: How to Deal with Plausible Values from International Large-Scale Assessments," Mathematics, MDPI, vol. 9(13), pages 1-16, July.
    4. Aparicio, Juan & Ortiz, Lidia & Santín, Daniel, 2021. "Comparing group performance over time through the Luenberger productivity indicator: An application to school ownership in European countries," European Journal of Operational Research, Elsevier, vol. 294(2), pages 651-672.
    5. López-Torres, Laura & Prior, Diego, 2022. "Long-term efficiency of public service provision in a context of budget restrictions. An application to the education sector," Socio-Economic Planning Sciences, Elsevier, vol. 81(C).
    6. Alexander Arévalo S & Víctor Giménez G & Diego Prior J, 2022. "Análisis de eficiencia en educación: una aplicación del método StoNED," Revista Desarrollo y Sociedad, Universidad de los Andes,Facultad de Economía, CEDE, vol. 92(2), pages 45-91, October.
    7. Aparicio, Juan & López-Torres, Laura & Santín, Daniel, 2018. "Economic crisis and public education. A productivity analysis using a Hicks-Moorsteen index," Economic Modelling, Elsevier, vol. 71(C), pages 34-44.
    8. Cordero, Jose Manuel & Polo, Cristina & Simancas, Rosa, 2022. "Assessing the efficiency of secondary schools: Evidence from OECD countries participating in PISA 2015," Socio-Economic Planning Sciences, Elsevier, vol. 81(C).
    9. Valentin Zelenyuk, 2019. "Data Envelopment Analysis and Business Analytics: The Big Data Challenges and Some Solutions," CEPA Working Papers Series WP072019, School of Economics, University of Queensland, Australia.
    10. Amir Moradi-Motlagh & Ali Emrouznejad, 2022. "The origins and development of statistical approaches in non-parametric frontier models: a survey of the first two decades of scholarly literature (1998–2020)," Annals of Operations Research, Springer, vol. 318(1), pages 713-741, November.
    11. Ali Ebrahimnejad & Madjid Tavana & Seyed Hadi Nasseri & Omid Gholami, 2019. "A New Method for Solving Dual DEA Problems with Fuzzy Stochastic Data," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 18(01), pages 147-170, January.
    12. Arbona, Alexei & Giménez, Víctor & López-Estrada, Sebastián & Prior, Diego, 2022. "Efficiency and quality in Colombian education: An application of the metafrontier Malmquist-Luenberger productivity index," Socio-Economic Planning Sciences, Elsevier, vol. 79(C).
    13. Delprato, Marcos & Antequera, Germán, 2021. "School efficiency in low and middle income countries: An analysis based on PISA for development learning survey," International Journal of Educational Development, Elsevier, vol. 80(C).
    14. Liu, John S. & Lu, Louis Y.Y. & Lu, Wen-Min, 2016. "Research fronts in data envelopment analysis," Omega, Elsevier, vol. 58(C), pages 33-45.
    15. Paolo Liberati & Raffaele Lagravinese & Giuliano Resce, 2017. "How Does Economic Social And Cultural Status Affect The Efficiency Of Educational Attainments? A Comparative Analysis On Pisa Results," Departmental Working Papers of Economics - University 'Roma Tre' 0217, Department of Economics - University Roma Tre.
    16. Vittadini, Giorgio & Sturaro, Caterina & Folloni, Giuseppe, 2022. "Non-Cognitive Skills and Cognitive Skills to measure school efficiency," Socio-Economic Planning Sciences, Elsevier, vol. 81(C).
    17. Adel Hatami-Marbini & Zahra Ghelej Beigi & Jens Leth Hougaard & Kobra Gholami, 2014. "Estimating Returns to Scale in Imprecise Data Envelopment Analysis," MSAP Working Paper Series 07_2014, University of Copenhagen, Department of Food and Resource Economics.
    18. Paola Azar & Gabriela Sicilia, 2021. "An assessment of pupil and school performance in public primary education in Uruguay," Documentos de Trabajo (working papers) 21-22, Instituto de Economía - IECON.
    19. Pourmahmoud, Jafar & Bagheri, Narges, 2023. "Uncertain Malmquist productivity index: An application to evaluate healthcare systems during COVID-19 pandemic," Socio-Economic Planning Sciences, Elsevier, vol. 87(PA).
    20. Cordero Ferrera, Jose Manuel & Alonso Morán, Edurne & Nuño Solís, Roberto & Orueta, Juan F. & Souto Arce, Regina, 2013. "Efficiency assessment of primary care providers: A conditional nonparametric approach," MPRA Paper 51926, University Library of Munich, Germany.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:soceps:v:68:y:2019:i:c:s0038012118302684. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/seps .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.