IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v72y2007i1d10.1007_s11192-007-1712-1.html
   My bibliography  Save this article

Irreproducibility of the results of the Shanghai academic ranking of world universities

Author

Listed:
  • Răzvan V. Florian

    (Ad Astra Association of Romanian Scientists
    Center for Cognitive and Neural Studies (Coneural))

Abstract

I discuss the difficulties that I encountered in reproducing the results of the Shanghai ranking of world universities. In the Shanghai ranking, the dependence between the score for the SCI indicator and the weighted number of considered articles obeys a power law, instead of the proportional dependence that is suggested by the official methodology of the ranking. Discrepancies from proportionality are also found in some of the scores for the N&S and Size indicators. This shows that the results of the Shanghai ranking cannot be reproduced, given raw data and the public methodology of the ranking.

Suggested Citation

  • Răzvan V. Florian, 2007. "Irreproducibility of the results of the Shanghai academic ranking of world universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 72(1), pages 25-32, July.
    • Handle: RePEc:spr:scient:v:72:y:2007:i:1:d:10.1007_s11192-007-1712-1
    DOI: 10.1007/s11192-007-1712-1
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-007-1712-1
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-007-1712-1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Anthony F. J. van Raan, 2005. "Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods," Scientometrics, Springer;Akadémiai Kiadó, vol. 62(1), pages 133-143, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Domingo Docampo & Lawrence Cram, 2019. "Highly cited researchers: a moving target," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(3), pages 1011-1025, March.
    2. Ludo Waltman & Clara Calero-Medina & Joost Kosten & Ed C.M. Noyons & Robert J.W. Tijssen & Nees Jan Eck & Thed N. Leeuwen & Anthony F.J. Raan & Martijn S. Visser & Paul Wouters, 2012. "The Leiden ranking 2011/2012: Data collection, indicators, and interpretation," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(12), pages 2419-2432, December.
    3. Paul F. Skilton, 2009. "Does the human capital of teams of natural science authors predict citation frequency?," Scientometrics, Springer;Akadémiai Kiadó, vol. 78(3), pages 525-542, March.
    4. Daraio, Cinzia & Bonaccorsi, Andrea & Simar, Léopold, 2015. "Rankings and university performance: A conditional multidimensional approach," European Journal of Operational Research, Elsevier, vol. 244(3), pages 918-930.
    5. Domingo Docampo, 2013. "Reproducibility of the Shanghai academic ranking of world universities results," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(2), pages 567-587, February.
    6. Jean-Charles Billaut & Denis Bouyssou & Philippe Vincke, 2010. "Should you believe in the Shanghai ranking?," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(1), pages 237-263, July.
    7. Jean-Charles Billaut & Denis Bouyssou & Philippe Vincke, 2009. "Should you believe in the Shanghai ranking? An MCDM view," Working Papers hal-00877050, HAL.
    8. Antonio Fernández-Cano & Elvira Curiel-Marin & Manuel Torralbo-Rodríguez & Mónica Vallejo-Ruiz, 2018. "Questioning the Shanghai Ranking methodology as a tool for the evaluation of universities: an integrative review," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 2069-2083, September.
    9. Veljko Jeremic & Milica Bulajic & Milan Martic & Zoran Radojicic, 2011. "A fresh approach to evaluating the academic ranking of world universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(3), pages 587-596, June.
    10. Viktor Koksharov, 2014. "Development Dynamics of the Higher Education System in Russia," Economy of region, Centre for Economic Security, Institute of Economics of Ural Branch of Russian Academy of Sciences, vol. 1(4), pages 30-44.
    11. Viktor Koksharov & Daniil Sandler & Sergey Kadochnikov & Dmitrii Tolmachev, 2012. "Research and development potential of the Russian higher education institutions," Economy of region, Centre for Economic Security, Institute of Economics of Ural Branch of Russian Academy of Sciences, vol. 1(3), pages 33-46.
    12. Lucia Buzzigoli & Antonio Giusti & Alessandro Viviani, 2010. "The Evaluation of University Departments. A Case Study for Firenze," International Advances in Economic Research, Springer;International Atlantic Economic Society, vol. 16(1), pages 24-38, February.
    13. Ulrich Schmoch & Habib M. Fardoun & Abdulfattah S. Mashat, 2016. "Establishing a World-Class University in Saudi Arabia: intended and unintended effects," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 1191-1207, November.
    14. Poh Kam Wong & Annette Singh, 2010. "University patenting activities and their link to the quantity and quality of scientific publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 83(1), pages 271-294, April.
    15. Osmo Kivinen & Juha Hedman & Päivi Kaipainen, 2013. "Productivity analysis of research in Natural Sciences, Technology and Clinical Medicine: an input–output model applied in comparison of Top 300 ranked universities of 4 North European and 4 East Asian," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(2), pages 683-699, February.
    16. Leo Freyer, 2014. "Robust rankings," Scientometrics, Springer;Akadémiai Kiadó, vol. 100(2), pages 391-406, August.
    17. Enis Siniksaran & M. Hakan Satman, 2020. "WURS: a simulation software for university rankings—software review," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(1), pages 701-717, January.
    18. D. Docampo & D. Egret & L. Cram, 2015. "The effect of university mergers on the Shanghai ranking," Scientometrics, Springer;Akadémiai Kiadó, vol. 104(1), pages 175-191, July.
    19. Daniele Fanelli, 2010. "Do Pressures to Publish Increase Scientists' Bias? An Empirical Support from US States Data," PLOS ONE, Public Library of Science, vol. 5(4), pages 1-7, April.
    20. Billaut, Jean-Charles & Bouyssou, Denis & Vincke, Philippe, 2010. "Faut-il croire le classement de Shangaï ?," Revue de la Régulation - Capitalisme, institutions, pouvoirs, Association Recherche et Régulation, vol. 8.
    21. Thum, Anna-Elisabeth & Roth,Felix, 2010. "The Key Role of Education in the Europe 2020 Strategy," CEPS Papers 3827, Centre for European Policy Studies.
    22. Domingo Docampo, 2011. "On using the Shanghai ranking to assess the research performance of university systems," Scientometrics, Springer;Akadémiai Kiadó, vol. 86(1), pages 77-92, January.
    23. O. Mryglod & R. Kenna & Yu. Holovatch & B. Berche, 2013. "Comparison of a citation-based indicator and peer review for absolute and specific measures of research-group excellence," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(3), pages 767-777, December.
    24. Alan Peter Matthews, 2012. "South African universities in world rankings," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(3), pages 675-695, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Rabishankar Giri & Sabuj Kumar Chaudhuri, 2021. "Ranking journals through the lens of active visibility," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(3), pages 2189-2208, March.
    2. Domingo Docampo & Lawrence Cram, 2019. "Highly cited researchers: a moving target," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(3), pages 1011-1025, March.
    3. Mehdi Rhaiem & Nabil Amara, 2020. "Determinants of research efficiency in Canadian business schools: evidence from scholar-level data," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 53-99, October.
    4. Daraio, Cinzia & Bonaccorsi, Andrea & Simar, Léopold, 2015. "Rankings and university performance: A conditional multidimensional approach," European Journal of Operational Research, Elsevier, vol. 244(3), pages 918-930.
    5. Daraio, Cinzia & Bonaccorsi, Andrea & Geuna, Aldo & Lepori, Benedetto & Bach, Laurent & Bogetoft, Peter & F. Cardoso, Margarida & Castro-Martinez, Elena & Crespi, Gustavo & de Lucio, Ignacio Fernandez, 2011. "The European university landscape: A micro characterization based on evidence from the Aquameth project," Research Policy, Elsevier, vol. 40(1), pages 148-164, February.
    6. Gennaro Guida, 2018. "Italian Economics Departments’ Scientific Research Performance: Comparison between VQR and ASN Methodologies," International Journal of Business and Management, Canadian Center of Science and Education, vol. 13(9), pages 182-182, August.
    7. Saisana, Michaela & d'Hombres, Béatrice & Saltelli, Andrea, 2011. "Rickety numbers: Volatility of university rankings and policy implications," Research Policy, Elsevier, vol. 40(1), pages 165-177, February.
    8. Anastasios Zopiatis & Antonis L. Theocharous & Panayiotis Constanti, 2015. "‘The past is prologue to the future’: an introspective view of hospitality and tourism research," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(2), pages 1731-1753, February.
    9. Auranen, Otto & Nieminen, Mika, 2010. "University research funding and publication performance--An international comparison," Research Policy, Elsevier, vol. 39(6), pages 822-834, July.
    10. Ruimin Ma & Chaoqun Ni & Junping Qiu, 2008. "Scientific research competitiveness of world universities in computer science," Scientometrics, Springer;Akadémiai Kiadó, vol. 76(2), pages 245-260, August.
    11. Giovanni Abramo & Ciriaco Andrea D’Angelo & Flavia Di Costa, 2011. "A national-scale cross-time analysis of university research performance," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(2), pages 399-413, May.
    12. Varshavsky, Alexander & Komkina, Tatiana, 2011. "Analysis of two main university rankings," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 23(3), pages 57-78.
    13. Abramo, Giovanni & D’Angelo, Ciriaco Andrea, 2015. "Evaluating university research: Same performance indicator, different rankings," Journal of Informetrics, Elsevier, vol. 9(3), pages 514-525.
    14. Anthony F. J. Raan, 2006. "Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups," Scientometrics, Springer;Akadémiai Kiadó, vol. 67(3), pages 491-502, June.
    15. Lipeng Fan & Yuefen Wang & Shengchun Ding & Binbin Qi, 2020. "Productivity trends and citation impact of different institutional collaboration patterns at the research units’ level," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 1179-1196, November.
    16. Amanda H Goodall, 2005. "Should top universities be led by top researchers, and are they?," General Economics and Teaching 0510003, University Library of Munich, Germany.
    17. Mallig, Nicolai, 2010. "A relational database for bibliometric analysis," Journal of Informetrics, Elsevier, vol. 4(4), pages 564-580.
    18. J. W. Fedderke, 2013. "The objectivity of national research foundation peer review in South Africa assessed against bibliometric indexes," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(2), pages 177-206, November.
    19. Yang Zhang & Yu Xiao & Jun Wu & Xin Lu, 2021. "Comprehensive world university ranking based on ranking aggregation," Computational Statistics, Springer, vol. 36(2), pages 1139-1152, June.
    20. Fernanda Morillo & Ignacio Santabárbara & Javier Aparicio, 2013. "The automatic normalisation challenge: detailed addresses identification," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(3), pages 953-966, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:72:y:2007:i:1:d:10.1007_s11192-007-1712-1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.