IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v129y2024i6d10.1007_s11192-024-05039-7.html
   My bibliography  Save this article

The use of ChatGPT to find similar institutions for institutional benchmarking

Author

Listed:
  • Lutz Bornmann

    (Administrative Headquarters of the Max Planck Society)

  • Benedetto Lepori

    (Università Della Svizzera Italiana)

Abstract

In evaluative bibliometrics and higher education studies, one is frequently confronted with the task of comparing institutions with similar institutions. In this Letter to the Editor, a simple approach is discussed which applies ChatGPT. Although the approach seems to produce promising results (tested with an example at the level of research institute and of a university), it is necessary to investigate it systematically based on a sample including many institutions before it should be applied in research evaluation. The challenge in systematic investigations is that ChatGPT provides the user with different answers on the sane request (missing reliability).

Suggested Citation

  • Lutz Bornmann & Benedetto Lepori, 2024. "The use of ChatGPT to find similar institutions for institutional benchmarking," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(6), pages 3593-3598, June.
    • Handle: RePEc:spr:scient:v:129:y:2024:i:6:d:10.1007_s11192-024-05039-7
    DOI: 10.1007/s11192-024-05039-7
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-024-05039-7
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-024-05039-7?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Daraio, Cinzia & Bonaccorsi, Andrea & Simar, Léopold, 2015. "Rankings and university performance: A conditional multidimensional approach," European Journal of Operational Research, Elsevier, vol. 244(3), pages 918-930.
    2. Qi Wang & Tobias Jeppsson, 2022. "Identifying benchmark units for research management and evaluation," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 7557-7574, December.
    3. Nicolas Carayol & Ghislaine Filliatreau & Agenor Lahatte, 2012. "Reference classes: a tool for benchmarking universities’ research," Scientometrics, Springer;Akadémiai Kiadó, vol. 93(2), pages 351-371, November.
    4. E.C.M. Noyons & H.F. Moed & M. Luwel, 1999. "Combining mapping and citation analysis for evaluative bibliometric purposes: A bibliometric study," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 50(2), pages 115-131.
    5. Lutz Bornmann & Felix de Moya Anegón & Rüdiger Mutz, 2013. "Do Universities or Research Institutions With a Specific Subject Profile Have an Advantage or a Disadvantage in Institutional Rankings? A Latent Class Analysis With Data From the SCImago Ranking," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(11), pages 2310-2316, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Klaus Wohlrabe & Felix de Moya Anegon & Lutz Bornmann, 2018. "How efficiently produce elite US universities highly cited papers? A case study based on input and output data," ifo Working Paper Series 264, ifo Institute - Leibniz Institute for Economic Research at the University of Munich.
    2. Qi Wang & Tobias Jeppsson, 2022. "Identifying benchmark units for research management and evaluation," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 7557-7574, December.
    3. Klaus Wohlrabe & Félix de Moya Anegon & Lutz Bornmann, 2019. "How Efficiently Do Elite US Universities Produce Highly Cited Papers?," Publications, MDPI, vol. 7(1), pages 1-15, January.
    4. Gaviria-Marin, Magaly & Merigó, José M. & Baier-Fuentes, Hugo, 2019. "Knowledge management: A global examination based on bibliometric analysis," Technological Forecasting and Social Change, Elsevier, vol. 140(C), pages 194-220.
    5. Gallego-Losada, María-Jesús & Montero-Navarro, Antonio & García-Abajo, Elisa & Gallego-Losada, Rocío, 2023. "Digital financial inclusion. Visualizing the academic literature," Research in International Business and Finance, Elsevier, vol. 64(C).
    6. Mehdi Rhaiem & Nabil Amara, 2020. "Determinants of research efficiency in Canadian business schools: evidence from scholar-level data," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 53-99, October.
    7. Daraio, Cinzia & Bonaccorsi, Andrea & Simar, Léopold, 2015. "Rankings and university performance: A conditional multidimensional approach," European Journal of Operational Research, Elsevier, vol. 244(3), pages 918-930.
    8. Cinzia Daraio & Simone Di Leo & Loet Leydesdorff, 2022. "Using the Leiden Rankings as a Heuristics: Evidence from Italian universities in the European landscape," LEM Papers Series 2022/08, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    9. Pan Zhang & Yongjun Du & Sijie Han & Qingan Qiu, 2022. "Global Progress in Oil and Gas Well Research Using Bibliometric Analysis Based on VOSviewer and CiteSpace," Energies, MDPI, vol. 15(15), pages 1-27, July.
    10. Degl’Innocenti, Marta & Matousek, Roman & Tzeremes, Nickolaos G., 2019. "The interconnections of academic research and universities’ “third mission”: Evidence from the UK," Research Policy, Elsevier, vol. 48(9), pages 1-1.
    11. Minviel, Jean Joseph & De Witte, Kristof, 2017. "The influence of public subsidies on farm technical efficiency: A robust conditional nonparametric approach," European Journal of Operational Research, Elsevier, vol. 259(3), pages 1112-1120.
    12. Wang, Derek D., 2019. "Performance-based resource allocation for higher education institutions in China," Socio-Economic Planning Sciences, Elsevier, vol. 65(C), pages 66-75.
    13. Andrej Kastrin & Dimitar Hristovski, 2021. "Scientometric analysis and knowledge mapping of literature-based discovery (1986–2020)," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1415-1451, February.
    14. Zorica Lazić & Aleksandar Đorđević & Albina Gazizulina, 2021. "Improvement of Quality of Higher Education Institutions as a Basis for Improvement of Quality of Life," Sustainability, MDPI, vol. 13(8), pages 1-27, April.
    15. Avenali, Alessandro & Daraio, Cinzia & Di Leo, Simone & Wolszczak-Derlacz, Joanna, 2024. "Heterogeneity of national accounting systems, world-class universities and financial resources: What are the links?," Journal of Informetrics, Elsevier, vol. 18(2).
    16. Félix Moya-Anegón & Benjamín Vargas-Quesada & Victor Herrero-Solana & Zaida Chinchilla-Rodríguez & Elena Corera-Álvarez & Francisco J. Munoz-Fernández, 2004. "A new technique for building maps of large scientific domains based on the cocitation of classes and categories," Scientometrics, Springer;Akadémiai Kiadó, vol. 61(1), pages 129-145, September.
    17. Huixin Wang & Jing Xie & Shixian Luo & Duy Thong Ta & Qian Wang & Jiao Zhang & Daer Su & Katsunori Furuya, 2023. "Exploring the Interplay between Landscape Planning and Human Well-Being: A Scientometric Review," Land, MDPI, vol. 12(7), pages 1-24, June.
    18. Bădin, Luiza & Daraio, Cinzia & Simar, Léopold, 2019. "A bootstrap approach for bandwidth selection in estimating conditional efficiency measures," European Journal of Operational Research, Elsevier, vol. 277(2), pages 784-797.
    19. Klaus Wohlrabe & Elisabeth Friedrich, 2017. "The efficiency of economics departments reconsidered," Economics Bulletin, AccessEcon, vol. 37(3), pages 1602-1611.
    20. John Mingers & Jesse R. O’Hanley & Musbaudeen Okunola, 2017. "Using Google Scholar institutional level data to evaluate the quality of university research," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(3), pages 1627-1643, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:129:y:2024:i:6:d:10.1007_s11192-024-05039-7. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.