IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v115y2018i1d10.1007_s11192-018-2666-1.html
   My bibliography  Save this article

University rankings: What do they really show?

Author

Listed:
  • Jill Johnes

    (University of Huddersfield)

Abstract

University rankings as developed by the media are used by many stakeholders in higher education: students looking for university places; academics looking for university jobs; university managers who need to maintain standing in the competitive arena of student recruitment; and governments who want to know that public funds spent on universities are delivering a world class higher education system. Media rankings deliberately draw attention to the performance of each university relative to all others, and as such they are undeniably simple to use and interpret. But one danger is that they are potentially open to manipulation and gaming because many of the measures underlying the rankings are under the control of the institutions themselves. This paper examines media rankings (constructed from an amalgamation of variables representing performance across numerous dimensions) to reveal the problems with using a composite index to reflect overall performance. It ends with a proposal for an alternative methodology which leads to groupings rather than point estimates.

Suggested Citation

  • Jill Johnes, 2018. "University rankings: What do they really show?," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 585-606, April.
    • Handle: RePEc:spr:scient:v:115:y:2018:i:1:d:10.1007_s11192-018-2666-1
    DOI: 10.1007/s11192-018-2666-1
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-018-2666-1
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-018-2666-1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Muller, Seán M, 2017. "Academics as rent seekers: distorted incentives in higher education, with reference to the South African case," International Journal of Educational Development, Elsevier, vol. 52(C), pages 58-67.
    2. Johnes, Jill, 2015. "Operational Research in education," European Journal of Operational Research, Elsevier, vol. 243(3), pages 683-696.
    3. Ronald G. Ehrenberg, 2012. "American Higher Education in Transition," Journal of Economic Perspectives, American Economic Association, vol. 26(1), pages 193-216, Winter.
    4. Popov, Sergey V. & Bernhardt, Dan, 2010. "University Competition, Grading Standards and Grade Inflation," MPRA Paper 26461, University Library of Munich, Germany.
    5. Johnes, Jill, 1996. "Performance assessment in higher education in Britain," European Journal of Operational Research, Elsevier, vol. 89(1), pages 18-33, February.
    6. Mika Nieminen, 2009. "Influence of research funding and science policy on university research performance: A comparison of five countries," Science and Public Policy, Oxford University Press, vol. 36(6), pages 419-430, July.
    7. Michela Nardo & Michaela Saisana & Andrea Saltelli & Stefano Tarantola & Anders Hoffman & Enrico Giovannini, 2005. "Handbook on Constructing Composite Indicators: Methodology and User Guide," OECD Statistics Working Papers 2005/3, OECD Publishing.
    8. Johnes, Geraint, 1992. "Performance Indicators in Higher Education: A Survey of Recent Work," Oxford Review of Economic Policy, Oxford University Press and Oxford Review of Economic Policy Limited, vol. 8(2), pages 19-34, Summer.
    9. Kelchtermans, Stijn & Verboven, Frank, 2010. "Program duplication in higher education is not necessarily bad," Journal of Public Economics, Elsevier, vol. 94(5-6), pages 397-409, June.
    10. Laurens Cherchye & Willem Moesen & Nicky Rogge & Tom Puyenbroeck, 2007. "An Introduction to ‘Benefit of the Doubt’ Composite Indicators," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 82(1), pages 111-145, May.
    11. De Fraja, Gianni & Valbonesi, Paola, 2012. "The design of the university system," Journal of Public Economics, Elsevier, vol. 96(3), pages 317-330.
    12. Charnes, A. & Cooper, W. W. & Rhodes, E., 1978. "Measuring the efficiency of decision making units," European Journal of Operational Research, Elsevier, vol. 2(6), pages 429-444, November.
    13. Saisana, Michaela & d'Hombres, Béatrice & Saltelli, Andrea, 2011. "Rickety numbers: Volatility of university rankings and policy implications," Research Policy, Elsevier, vol. 40(1), pages 165-177, February.
    14. repec:bla:ecinqu:v:51:y:2013:i:3:p:1764-1778 is not listed on IDEAS
    15. Hicks, Diana, 2012. "Performance-based university research funding systems," Research Policy, Elsevier, vol. 41(2), pages 251-261.
    16. Marie-Laure Bougnol & José Dulá, 2006. "Validating DEA as a ranking tool: An application of DEA to assess performance in higher education," Annals of Operations Research, Springer, vol. 145(1), pages 339-365, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Buckley, Ralf, 2019. "Tourism publications as newly tradeable commodities: Academic performance, prestige, power, competition, constraints and consents," Annals of Tourism Research, Elsevier, vol. 74(C), pages 121-133.
    2. Kristof De Witte & Laura López-Torres, 2017. "Efficiency in education: a review of literature and a way forward," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 68(4), pages 339-363, April.
    3. Milica Maricic & Jose A. Egea & Veljko Jeremic, 2019. "A Hybrid Enhanced Scatter Search—Composite I-Distance Indicator (eSS-CIDI) Optimization Approach for Determining Weights Within Composite Indicators," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 144(2), pages 497-537, July.
    4. Renato A. Villano & Carolyn‐Dung T. T. Tran, 2021. "Survey on technical efficiency in higher education: A meta‐fractional regression analysis," Pacific Economic Review, Wiley Blackwell, vol. 26(1), pages 110-135, February.
    5. Van Puyenbroeck, Tom & Rogge, Nicky, 2017. "Geometric mean quantity index numbers with Benefit-of-the-Doubt weights," European Journal of Operational Research, Elsevier, vol. 256(3), pages 1004-1014.
    6. Laurens CHERCHYE & Willem MOESEN & Nicky ROGGE & Tom VAN PUYENBROECK, 2009. "Constructing a knowledge economy composite indicator with imprecise data," Working Papers of Department of Economics, Leuven ces09.15, KU Leuven, Faculty of Economics and Business (FEB), Department of Economics, Leuven.
    7. Giménez, Víctor & Thieme, Claudio & Prior, Diego & Tortosa-Ausina, Emili, 2022. "Evaluation and determinants of preschool effectiveness in Chile," Socio-Economic Planning Sciences, Elsevier, vol. 81(C).
    8. Salvatore Greco & Alessio Ishizaka & Menelaos Tasiou & Gianpiero Torrisi, 2019. "On the Methodological Framework of Composite Indices: A Review of the Issues of Weighting, Aggregation, and Robustness," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 141(1), pages 61-94, January.
    9. Guironnet, Jean-Pascal & Peypoch, Nicolas, 2018. "The geographical efficiency of education and research: The ranking of U.S. universities," Socio-Economic Planning Sciences, Elsevier, vol. 62(C), pages 44-55.
    10. P. Zhou & B. Ang, 2009. "Comparing MCDA Aggregation Methods in Constructing Composite Indicators Using the Shannon-Spearman Measure," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 94(1), pages 83-96, October.
    11. Diogo Ferraz & Enzo B. Mariano & Daisy Rebelatto & Dominik Hartmann, 2020. "Linking Human Development and the Financial Responsibility of Regions: Combined Index Proposals Using Methods from Data Envelopment Analysis," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 150(2), pages 439-478, July.
    12. Maričić Milica & Bulajić Milica & Radojičić Zoran & Jeremić Veljko, 2016. "Multivariate approach to imposing additional constraints on the Benefit-of-the-Doubt model: The case of QS World University Rankings by Subject," Croatian Review of Economic, Business and Social Statistics, Sciendo, vol. 2(1), pages 1-14, September.
    13. Tom Puyenbroeck, 2018. "On the Output Orientation of the Benefit-of-the-Doubt-Model," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 139(2), pages 415-431, September.
    14. Ana Cárcaba & Eduardo González & Juan Ventura & Rubén Arrondo, 2017. "How Does Good Governance Relate to Quality of Life?," Sustainability, MDPI, vol. 9(4), pages 1-16, April.
    15. Tzeremes, Nickolaos & Halkos, George, 2010. "A DEA approach for measuring university departments’ efficiency," MPRA Paper 24029, University Library of Munich, Germany.
    16. Ana Cárcaba & Eduardo González & Juan Ventura, 2017. "Social Progress in Spanish Municipalities (2001–2011)," Applied Research in Quality of Life, Springer;International Society for Quality-of-Life Studies, vol. 12(4), pages 997-1019, December.
    17. Hatefi, S.M. & Torabi, S.A., 2010. "A common weight MCDA-DEA approach to construct composite indicators," Ecological Economics, Elsevier, vol. 70(1), pages 114-120, November.
    18. Omrani, Hashem & Valipour, Mahsa & Jafari Mamakani, Saeid, 2019. "Construct a composite indicator based on integrating Common Weight Data Envelopment Analysis and principal component analysis models: An application for finding development degree of provinces in Iran," Socio-Economic Planning Sciences, Elsevier, vol. 68(C).
    19. Sunil Kumar Jauhar & Millie Pant & Richa Dutt, 2018. "Performance measurement of an Indian higher education institute: a sustainable educational supply chain management perspective," International Journal of System Assurance Engineering and Management, Springer;The Society for Reliability, Engineering Quality and Operations Management (SREQOM),India, and Division of Operation and Maintenance, Lulea University of Technology, Sweden, vol. 9(1), pages 180-193, February.
    20. Eduardo González & Ana Cárcaba & Juan Ventura, 2018. "Weight Constrained DEA Measurement of the Quality of Life in Spanish Municipalities in 2011," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 136(3), pages 1157-1182, April.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:115:y:2018:i:1:d:10.1007_s11192-018-2666-1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.