IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v129y2024i6d10.1007_s11192-024-05039-7.html
   My bibliography  Save this article

The use of ChatGPT to find similar institutions for institutional benchmarking

Author

Listed:
  • Lutz Bornmann

    (Administrative Headquarters of the Max Planck Society)

  • Benedetto Lepori

    (Università Della Svizzera Italiana)

Abstract

In evaluative bibliometrics and higher education studies, one is frequently confronted with the task of comparing institutions with similar institutions. In this Letter to the Editor, a simple approach is discussed which applies ChatGPT. Although the approach seems to produce promising results (tested with an example at the level of research institute and of a university), it is necessary to investigate it systematically based on a sample including many institutions before it should be applied in research evaluation. The challenge in systematic investigations is that ChatGPT provides the user with different answers on the sane request (missing reliability).

Suggested Citation

  • Lutz Bornmann & Benedetto Lepori, 2024. "The use of ChatGPT to find similar institutions for institutional benchmarking," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(6), pages 3593-3598, June.
    • Handle: RePEc:spr:scient:v:129:y:2024:i:6:d:10.1007_s11192-024-05039-7
    DOI: 10.1007/s11192-024-05039-7
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-024-05039-7
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-024-05039-7?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. E.C.M. Noyons & H.F. Moed & M. Luwel, 1999. "Combining mapping and citation analysis for evaluative bibliometric purposes: A bibliometric study," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 50(2), pages 115-131.
    2. Daraio, Cinzia & Bonaccorsi, Andrea & Simar, Léopold, 2015. "Rankings and university performance: A conditional multidimensional approach," European Journal of Operational Research, Elsevier, vol. 244(3), pages 918-930.
    3. Nicolas Carayol & Ghislaine Filliatreau & Agenor Lahatte, 2012. "Reference classes: a tool for benchmarking universities’ research," Scientometrics, Springer;Akadémiai Kiadó, vol. 93(2), pages 351-371, November.
    4. Qi Wang & Tobias Jeppsson, 2022. "Identifying benchmark units for research management and evaluation," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 7557-7574, December.
    5. Lutz Bornmann & Felix de Moya Anegón & Rüdiger Mutz, 2013. "Do Universities or Research Institutions With a Specific Subject Profile Have an Advantage or a Disadvantage in Institutional Rankings? A Latent Class Analysis With Data From the SCImago Ranking," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(11), pages 2310-2316, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Giulio Giacomo Cantone, 2024. "How to measure interdisciplinary research? A systemic design for the model of measurement," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(8), pages 4937-4982, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Klaus Wohlrabe & Felix de Moya Anegon & Lutz Bornmann, 2018. "How efficiently produce elite US universities highly cited papers? A case study based on input and output data," ifo Working Paper Series 264, ifo Institute - Leibniz Institute for Economic Research at the University of Munich.
    2. Qi Wang & Tobias Jeppsson, 2022. "Identifying benchmark units for research management and evaluation," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 7557-7574, December.
    3. Klaus Wohlrabe & Félix de Moya Anegon & Lutz Bornmann, 2019. "How Efficiently Do Elite US Universities Produce Highly Cited Papers?," Publications, MDPI, vol. 7(1), pages 1-15, January.
    4. Piñeiro-Chousa, Juan & López-Cabarcos, M. Ángeles & Romero-Castro, Noelia María & Pérez-Pico, Ada María, 2020. "Innovation, entrepreneurship and knowledge in the business scientific field: Mapping the research front," Journal of Business Research, Elsevier, vol. 115(C), pages 475-485.
    5. Gaviria-Marin, Magaly & Merigó, José M. & Baier-Fuentes, Hugo, 2019. "Knowledge management: A global examination based on bibliometric analysis," Technological Forecasting and Social Change, Elsevier, vol. 140(C), pages 194-220.
    6. Gallego-Losada, María-Jesús & Montero-Navarro, Antonio & García-Abajo, Elisa & Gallego-Losada, Rocío, 2023. "Digital financial inclusion. Visualizing the academic literature," Research in International Business and Finance, Elsevier, vol. 64(C).
    7. P. Ebin Babu & Riya Mary, 2022. "Exploring the Research Trends in Green Tax: Bibliometric Analysis," International Journal of Energy Economics and Policy, Econjournals, vol. 12(6), pages 157-162, November.
    8. Mehdi Rhaiem & Nabil Amara, 2020. "Determinants of research efficiency in Canadian business schools: evidence from scholar-level data," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 53-99, October.
    9. Daraio, Cinzia & Simar, Leopold & Wilson, Paul, 2019. "Quality and its impact on efficiency," LIDAM Discussion Papers ISBA 2019004, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    10. Daraio, Cinzia & Bonaccorsi, Andrea & Simar, Léopold, 2015. "Rankings and university performance: A conditional multidimensional approach," European Journal of Operational Research, Elsevier, vol. 244(3), pages 918-930.
    11. Ming Tang & Huchang Liao & Zhengjun Wan & Enrique Herrera-Viedma & Marc A. Rosen, 2018. "Ten Years of Sustainability (2009 to 2018): A Bibliometric Overview," Sustainability, MDPI, vol. 10(5), pages 1-21, May.
    12. Cinzia Daraio & Simone Di Leo & Loet Leydesdorff, 2022. "Using the Leiden Rankings as a Heuristics: Evidence from Italian universities in the European landscape," LEM Papers Series 2022/08, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    13. Reza Naghizadeh & Shaban Elahi & Manoochehr Manteghi & Sepehr Ghazinoory & Marina Ranga, 2015. "Through the magnifying glass: an analysis of regional innovation models based on co-word and meta-synthesis methods," Quality & Quantity: International Journal of Methodology, Springer, vol. 49(6), pages 2481-2505, November.
    14. Sergio Pardo-Jaramillo & Andrés Muñoz-Villamizar & Ignacio Osuna & Rolando Roncancio, 2020. "Mapping Research on Customer Centricity and Sustainable Organizations," Sustainability, MDPI, vol. 12(19), pages 1-18, September.
    15. Marianna Mauro & Monica Giancotti & Giovanna Talarico, 2017. "Mapping the field: A bibliometric analysis of accountability literature in healthcare," MECOSAN, FrancoAngeli Editore, vol. 2017(101), pages 7-30.
    16. Arbrie Jashari & Victor Tiberius & Marina Dabić, 2022. "Tracing the progress of scenario research in business and management," Futures & Foresight Science, John Wiley & Sons, vol. 4(2), June.
    17. Pan Zhang & Yongjun Du & Sijie Han & Qingan Qiu, 2022. "Global Progress in Oil and Gas Well Research Using Bibliometric Analysis Based on VOSviewer and CiteSpace," Energies, MDPI, vol. 15(15), pages 1-27, July.
    18. Weisheng Chiu & Thomas Chun Man Fan & Sang-Back Nam & Ping-Hung Sun, 2021. "Knowledge Mapping and Sustainable Development of eSports Research: A Bibliometric and Visualized Analysis," Sustainability, MDPI, vol. 13(18), pages 1-17, September.
    19. Gallego-Losada, Rocío & Montero-Navarro, Antonio & Rodríguez-Sánchez, José-Luis & González-Torres, Thais, 2022. "Retirement planning and financial literacy, at the crossroads. A bibliometric analysis," Finance Research Letters, Elsevier, vol. 44(C).
    20. Degl’Innocenti, Marta & Matousek, Roman & Tzeremes, Nickolaos G., 2019. "The interconnections of academic research and universities’ “third mission”: Evidence from the UK," Research Policy, Elsevier, vol. 48(9), pages 1-1.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:129:y:2024:i:6:d:10.1007_s11192-024-05039-7. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.