IDEAS home Printed from https://ideas.repec.org/a/bla/buecrs/v58y2006i2p75-104.html
   My bibliography  Save this article

Measuring Efficiency: A Comparison of Multilevel Modelling and Data Envelopment Analysis in the Context of Higher Education

Author

Listed:
  • Jill Johnes

Abstract

Data envelopment analysis (DEA) and multilevel modelling (MLM) are applied to a data set of 54,564 graduates from UK universities in 1993 to assess whether the choice of technique affects the measurement of universities’ performance. A methodology developed by Thanassoulis and Portela (2002; Education Economics, 10(2), pp. 183–207) allows each individual's DEA efficiency score to be decomposed into two components: one attributable to the university at which the student studied and the other attributable to the individual student. From the former component, a measure of each institution's teaching efficiency is derived and compared to the university effects from various multilevel models. The comparisons are made within four broad subjects: pure science, applied science, social science and arts. The results show that the rankings of universities derived from the DEA efficiencies which measure the universities’ own performance (i.e., having excluded the efforts of the individuals) are not strongly correlated with the university rankings derived from the university effects of the multilevel models. The data were also used to perform a university‐level DEA. The university efficiency scores derived from these DEAs are largely unrelated to the scores from the individual‐level DEAs, confirming a result from a smaller data set (Johnes, 2006a; European Journal of Operational Research, forthcoming). However, the university‐level DEAs provide efficiency scores which are generally strongly related to the university effects of the multilevel models.

Suggested Citation

  • Jill Johnes, 2006. "Measuring Efficiency: A Comparison of Multilevel Modelling and Data Envelopment Analysis in the Context of Higher Education," Bulletin of Economic Research, Wiley Blackwell, vol. 58(2), pages 75-104, April.
    • Handle: RePEc:bla:buecrs:v:58:y:2006:i:2:p:75-104
    DOI: 10.1111/j.0307-3378.2006.00238.x
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/j.0307-3378.2006.00238.x
    Download Restriction: no
    File URL: https://libkey.io/10.1111/j.0307-3378.2006.00238.x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Forsund, Finn R. & Lovell, C. A. Knox & Schmidt, Peter, 1980. "A survey of frontier production functions and of their relationship to efficiency measurement," Journal of Econometrics, Elsevier, vol. 13(1), pages 5-25, May.
    2. Bratti, Massimiliano, 2002. "Does the choice of university matter?: a study of the differences across UK universities in life sciences students' degree performance," Economics of Education Review, Elsevier, vol. 21(5), pages 431-443, October.
    3. Philip Andrew Stevens, 2005. "A Stochastic Frontier Analysis of English and Welsh Universities," Education Economics, Taylor & Francis Journals, vol. 13(4), pages 355-374.
    4. Johnes, Jill, 1996. "Performance assessment in higher education in Britain," European Journal of Operational Research, Elsevier, vol. 89(1), pages 18-33, February.
    5. Massimiliano Bratti & Abigail McKnight & Robin Naylor & Jeremy Smith, 2004. "Higher education outcomes, graduate employment and university performance indicators," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 167(3), pages 475-496, August.
    6. Jeremy P. Smith & Robin A. Naylor, 2001. "Dropping out of university: A statistical analysis of the probability of withdrawal for UK university students," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 164(2), pages 389-405.
    7. Barrow, Michael M., 1991. "Measuring local education authority performance: A frontier approach," Economics of Education Review, Elsevier, vol. 10(1), pages 19-27, March.
    8. Antreas Athanassopoulos & Estelle Shale, 1997. "Assessing the Comparative Efficiency of Higher Education Institutions in the UK by the Means of Data Envelopment Analysis," Education Economics, Taylor & Francis Journals, vol. 5(2), pages 117-134.
    9. Gary Madden & Scott Savage & Steven Kemp, 1997. "Measuring Public Sector Efficiency: A Study of Economics Departments at Australian Universities," Education Economics, Taylor & Francis Journals, vol. 5(2), pages 153-168.
    10. Emmanuel Thanassoulis & Maria Da Conceicao & A. Silva Portela, 2002. "School Outcomes: Sharing the Responsibility Between Pupil and School1," Education Economics, Taylor & Francis Journals, vol. 10(2), pages 183-207.
    11. Deller, Steven C. & Rudnicki, Edward, 1993. "Production efficiency in elementary education: The case of Maine public schools," Economics of Education Review, Elsevier, vol. 12(1), pages 45-57, March.
    12. Izadi, Hooshang & Johnes, Geraint & Oskrochi, Reza & Crouchley, Robert, 2002. "Stochastic frontier estimation of a CES cost function: the case of higher education in Britain," Economics of Education Review, Elsevier, vol. 21(1), pages 63-71, February.
    13. Geraint Johnes, 1999. "The Management of Universities; Scottish Economic Society/Royal Bank of Scotland Annual Lecture, 1999," Scottish Journal of Political Economy, Scottish Economic Society, vol. 46(5), pages 505-522, November.
    14. Johnes, Jill, 2006. "Data envelopment analysis and its application to the measurement of efficiency in higher education," Economics of Education Review, Elsevier, vol. 25(3), pages 273-288, June.
    15. Korhonen, Pekka & Tainio, Risto & Wallenius, Jyrki, 2001. "Value efficiency analysis of academic research," European Journal of Operational Research, Elsevier, vol. 130(1), pages 121-132, April.
    16. Avkiran, Necmi K., 2001. "Investigating technical and scale efficiencies of Australian Universities through data envelopment analysis," Socio-Economic Planning Sciences, Elsevier, vol. 35(1), pages 57-80, March.
    17. Cuesta, Rafael A. & Orea, Luis, 2002. "Mergers and technical efficiency in Spanish savings banks: A stochastic distance function approach," Journal of Banking & Finance, Elsevier, vol. 26(12), pages 2231-2247.
    18. El-Mahgary, Sami & Lahdelma, Risto, 1995. "Data envelopment analysis: Visualizing the results," European Journal of Operational Research, Elsevier, vol. 83(3), pages 700-710, June.
    19. Johnes, Jill, 2006. "Measuring teaching efficiency in higher education: An application of data envelopment analysis to economics graduates from UK Universities 1993," European Journal of Operational Research, Elsevier, vol. 174(1), pages 443-456, October.
    20. Harvey Goldstein & David J. Spiegelhalter, 1996. "League Tables and Their Limitations: Statistical Issues in Comparisons of Institutional Performance," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 159(3), pages 385-409, May.
    21. Jeremy Smith & Robin Naylor, 2001. "Determinants of Degree Performance in UK Universities: A Statistical Analysis of the 1993 Student Cohort," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 63(1), pages 29-60, February.
    22. Fried, Harold O. & Lovell, C. A. Knox & Schmidt, Shelton S. (ed.), 1993. "The Measurement of Productive Efficiency: Techniques and Applications," OUP Catalogue, Oxford University Press, number 9780195072181.
    23. Smith, Jeremy & McKnight, Abigail & Naylor, Robin, 2000. "Graduate Employability: Policy and Performance in Higher Education in the UK," Economic Journal, Royal Economic Society, vol. 110(464), pages 382-411, June.
    24. Abbott, M. & Doucouliagos, C., 2003. "The efficiency of Australian universities: a data envelopment analysis," Economics of Education Review, Elsevier, vol. 22(1), pages 89-97, February.
    25. Aigner, Dennis & Lovell, C. A. Knox & Schmidt, Peter, 1977. "Formulation and estimation of stochastic frontier production function models," Journal of Econometrics, Elsevier, vol. 6(1), pages 21-37, July.
    26. Silva Portela, Maria Conceicao A. & Thanassoulis, Emmanuel, 2001. "Decomposing school and school-type efficiency," European Journal of Operational Research, Elsevier, vol. 132(2), pages 357-373, July.
    27. Harvey Goldstein & Sally Thomas, 1996. "Using Examination Results as Indicators of School and College Performance," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 159(1), pages 149-163, January.
    28. Johnes, Geraint, 1998. "The Costs of Multi-product Organizations and the Heuristic Evaluation of Industrial Structure," Socio-Economic Planning Sciences, Elsevier, vol. 32(3), pages 199-209, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kristof De Witte & Laura López-Torres, 2017. "Efficiency in education: a review of literature and a way forward," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 68(4), pages 339-363, April.
    2. Johnes, Jill, 2006. "Data envelopment analysis and its application to the measurement of efficiency in higher education," Economics of Education Review, Elsevier, vol. 25(3), pages 273-288, June.
    3. Mehdi Rhaiem, 2017. "Measurement and determinants of academic research efficiency: a systematic review of the evidence," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(2), pages 581-615, February.
    4. Johnes, Jill, 2015. "Operational Research in education," European Journal of Operational Research, Elsevier, vol. 243(3), pages 683-696.
    5. Johnes, Jill & Yu, Li, 2008. "Measuring the research performance of Chinese higher education institutions using data envelopment analysis," China Economic Review, Elsevier, vol. 19(4), pages 679-696, December.
    6. Guironnet, Jean-Pascal & Peypoch, Nicolas, 2018. "The geographical efficiency of education and research: The ranking of U.S. universities," Socio-Economic Planning Sciences, Elsevier, vol. 62(C), pages 44-55.
    7. Barra, Cristian & Lagravinese, Raffaele & Zotti, Roberto, 2018. "Does econometric methodology matter to rank universities? An analysis of Italian higher education system," Socio-Economic Planning Sciences, Elsevier, vol. 62(C), pages 104-120.
    8. Agasisti, Tommaso & Barra, Cristian & Zotti, Roberto, 2016. "Evaluating the efficiency of Italian public universities (2008–2011) in presence of (unobserved) heterogeneity," Socio-Economic Planning Sciences, Elsevier, vol. 55(C), pages 47-58.
    9. Martín Rivero, Raquel, 2007. "La eficiencia productiva en el ámbito universitario: aspectos claves para su evaluación/The Productive Efficiency in the University: Key Aspects for its Evaluation," Estudios de Economia Aplicada, Estudios de Economia Aplicada, vol. 25, pages 793-812, Diciembre.
    10. Laureti, Tiziana & Secondi, Luca & Biggeri, Luigi, 2014. "Measuring the efficiency of teaching activities in Italian universities: An information theoretic approach," Economics of Education Review, Elsevier, vol. 42(C), pages 147-164.
    11. Johnes, Jill, 2006. "Measuring teaching efficiency in higher education: An application of data envelopment analysis to economics graduates from UK Universities 1993," European Journal of Operational Research, Elsevier, vol. 174(1), pages 443-456, October.
    12. Tzeremes, Nickolaos & Halkos, George, 2010. "A DEA approach for measuring university departments’ efficiency," MPRA Paper 24029, University Library of Munich, Germany.
    13. Barra, Cristian & Lagravinese, Raffaele & Zotti, Roberto, 2015. "Explaining (in)efficiency in higher education: a comparison of parametric and non-parametric analyses to rank universities," MPRA Paper 67119, University Library of Munich, Germany.
    14. Manuel Salas-Velasco, 2020. "Measuring and explaining the production efficiency of Spanish universities using a non-parametric approach and a bootstrapped-truncated regression," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(2), pages 825-846, February.
    15. Elias Katsikas, 2009. "Elements and Symptoms of a Poor Higher Education system: Evidence from a Greek University," Discussion Paper Series 2009_17, Department of Economics, University of Macedonia, revised Dec 2009.
    16. Fatih KARASAÇ, 2020. "Efficiency Analysis of Turkish Higher Education Institutions: Stochastic Frontier Model," Sosyoekonomi Journal, Sosyoekonomi Society.
    17. Calogero Guccio & Marco Ferdinando Martorana & Isidoro Mazza, 2016. "Efficiency assessment and convergence in teaching and research in Italian public universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(3), pages 1063-1094, June.
    18. Mad Ithnin Salleh & Nurul Fadly Habidin & Abdul Halim Masnan & Nordin Mamat, 2017. "Estimating Technical Efficiency and Bootstrapping Malmquist Indices: Analysis of Malaysian Preschool Sector," International Journal of Academic Research in Business and Social Sciences, Human Resource Management Academic Research Society, International Journal of Academic Research in Business and Social Sciences, vol. 7(3), pages 440-457, March.
    19. Yaisawarng, Suthathip & Ng, Ying Chu, 2014. "The impact of higher education reform on research performance of Chinese universities," China Economic Review, Elsevier, vol. 31(C), pages 94-105.
    20. Lu, Wen-Min, 2012. "Intellectual capital and university performance in Taiwan," Economic Modelling, Elsevier, vol. 29(4), pages 1081-1089.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:buecrs:v:58:y:2006:i:2:p:75-104. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.blackwellpublishing.com/journal.asp?ref=0307-3378 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.