IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0253397.html
   My bibliography  Save this article

The h-index is no longer an effective correlate of scientific reputation

Author

Listed:
  • Vladlen Koltun
  • David Hafner

Abstract

The impact of individual scientists is commonly quantified using citation-based measures. The most common such measure is the h-index. A scientist’s h-index affects hiring, promotion, and funding decisions, and thus shapes the progress of science. Here we report a large-scale study of scientometric measures, analyzing millions of articles and hundreds of millions of citations across four scientific fields and two data platforms. We find that the correlation of the h-index with awards that indicate recognition by the scientific community has substantially declined. These trends are associated with changing authorship patterns. We show that these declines can be mitigated by fractional allocation of citations among authors, which has been discussed in the literature but not implemented at scale. We find that a fractional analogue of the h-index outperforms other measures as a correlate and predictor of scientific awards. Our results suggest that the use of the h-index in ranking scientists should be reconsidered, and that fractional allocation measures such as h-frac provide more robust alternatives.

Suggested Citation

  • Vladlen Koltun & David Hafner, 2021. "The h-index is no longer an effective correlate of scientific reputation," PLOS ONE, Public Library of Science, vol. 16(6), pages 1-16, June.
    • Handle: RePEc:plo:pone00:0253397
    DOI: 10.1371/journal.pone.0253397
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0253397
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0253397&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0253397?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. John Panaretos & Chrisovaladis Malesios, 2009. "Assessing scientific research performance and impact with single indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 81(3), pages 635-670, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Domenico A. Maisano & Luca Mastrogiacomo & Fiorenzo Franceschini, 2023. "Empirical evidence on the relationship between research and teaching in academia," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(8), pages 4475-4507, August.
    2. Alonso Rodríguez-Navarro & Ricardo Brito, 2022. "The link between countries’ economic and scientific wealth has a complex dependence on technological activity and research policy," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(5), pages 2871-2896, May.
    3. Ramona Bran & Laurentiu Tiru & Gabriela Grosseck & Carmen Holotescu & Laura Malita, 2021. "Learning from Each Other—A Bibliometric Review of Research on Information Disorders," Sustainability, MDPI, vol. 13(18), pages 1-39, September.
    4. Alejandro Agafonow & Marybel Perez, 2024. "When an A Is NOT an A in Academic Research, or How A-Journal List Metrics Inhibit Exploratory Behaviour in Academia," Journal of Interdisciplinary Economics, , vol. 36(1), pages 105-121, January.
    5. Gangan Prathap, 2021. "Letter to the editor: Dimensionless citation indicators for fractional counting," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(10), pages 8765-8769, October.
    6. Lauranne Chaignon & Domingo Docampo & Daniel Egret, 2023. "In search of a scientific elite: highly cited researchers (HCR) in France," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(10), pages 5801-5827, October.
    7. Yang, Alex Jie & Wu, Linwei & Zhang, Qi & Wang, Hao & Deng, Sanhong, 2023. "The k-step h-index in citation networks at the paper, author, and institution levels," Journal of Informetrics, Elsevier, vol. 17(4).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Deming Lin & Tianhui Gong & Wenbin Liu & Martin Meyer, 2020. "An entropy-based measure for the evolution of h index research," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2283-2298, December.
    2. Bornmann, Lutz & Mutz, Rüdiger & Daniel, Hans-Dieter, 2010. "The h index research output measurement: Two approaches to enhance its accuracy," Journal of Informetrics, Elsevier, vol. 4(3), pages 407-414.
    3. Zhenbin Yan & Qiang Wu & Xingchen Li, 2016. "Do Hirsch-type indices behave the same in assessing single publications? An empirical study of 29 bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1815-1833, December.
    4. Christoph Bartneck & Servaas Kokkelmans, 2011. "Detecting h-index manipulation through self-citation analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(1), pages 85-98, April.
    5. Panaretos, John & Malesios, Chrisovalandis, 2012. "Influential Mathematicians: Birth, Education and Affiliation," MPRA Paper 68046, University Library of Munich, Germany.
    6. Maziar Montazerian & Edgar Dutra Zanotto & Hellmut Eckert, 2019. "A new parameter for (normalized) evaluation of H-index: countries as a case study," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(3), pages 1065-1078, March.
    7. Lutz Bornmann & Werner Marx, 2014. "How to evaluate individual researchers working in the natural and life sciences meaningfully? A proposal of methods based on percentiles of citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(1), pages 487-509, January.
    8. C. O. S. Sorzano & J. Vargas & G. Caffarena-Fernández & A. Iriarte, 2014. "Comparing scientific performance among equals," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(3), pages 1731-1745, December.
    9. William Cabos & Juan Miguel Campanario, 2018. "Exploring the Hjif-Index, an Analogue to the H-Like Index for Journal Impact Factors," Publications, MDPI, vol. 6(2), pages 1-11, April.
    10. Bornmann, Lutz & Marx, Werner, 2012. "HistCite analysis of papers constituting the h index research front," Journal of Informetrics, Elsevier, vol. 6(2), pages 285-288.
    11. Eduardo A. Oliveira & Enrico A. Colosimo & Daniella R. Martelli & Isabel G. Quirino & Maria Christina L. Oliveira & Leonardo S. Lima & Ana Cristina Simões e Silva & Hercílio Martelli-Júnior, 2012. "Comparison of Brazilian researchers in clinical medicine: are criteria for ranking well-adjusted?," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(2), pages 429-443, February.
    12. Petridis, Konstantinos & Malesios, Chrisovalantis & Arabatzis, Garyfallos & Thanassoulis, Emmanuel, 2013. "Efficiency analysis of forestry journals: Suggestions for improving journals’ quality," Journal of Informetrics, Elsevier, vol. 7(2), pages 505-521.
    13. Roberto Todeschini, 2011. "The j-index: a new bibliometric index and multivariate comparisons between other common indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(3), pages 621-639, June.
    14. Lorna Wildgaard & Jesper W. Schneider & Birger Larsen, 2014. "A review of the characteristics of 108 author-level bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 125-158, October.
    15. Eduardo Araujo Oliveira & Roberto Peicots-Filho & Daniella Reis Martelli & Isabel Gomes Quirino & Maria Christina Lopes Oliveira & Mariana Guerra Duarte & Sergio Veloso Pinheiro & Enrico Antonio Colos, 2013. "Is there a correlation between journal impact factor and researchers’ performance? A study comprising the fields of clinical nephrology and neurosciences," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(2), pages 149-160, November.
    16. Edgar D. Zanotto & Vinicius Carvalho, 2021. "Article age- and field-normalized tools to evaluate scientific impact and momentum," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(4), pages 2865-2883, April.
    17. Burgos, Albert, 2010. "Measurable utility for scientific influence," UMUFAE Economics Working Papers 16347, DIGITUM. Universidad de Murcia.
    18. Ozlem Inanc & Onur Tuncer, 2011. "The effect of academic inbreeding on scientific effectiveness," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(3), pages 885-898, September.
    19. Giovanni Anania & Annarosa Caruso, 2013. "Two simple new bibliometric indexes to better evaluate research in disciplines where publications typically receive less citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(2), pages 617-631, August.
    20. Lyu, Haihua & Bu, Yi & Zhao, Zhenyue & Zhang, Jiarong & Li, Jiang, 2022. "Citation bias in measuring knowledge flow: Evidence from the web of science at the discipline level," Journal of Informetrics, Elsevier, vol. 16(4).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0253397. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.