IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0202712.html
   My bibliography  Save this article

Measuring researcher independence using bibliometric data: A proposal for a new performance indicator

Author

Listed:
  • Peter van den Besselaar
  • Ulf Sandström

Abstract

Bibliometric indicators are increasingly used to evaluate individual scientists–as is exemplified by the popularity of the many other publication and citation-based indicators used in evaluation. These indicators, however, cover at best some of the quality dimensions relevant for assessing a researcher: productivity and impact. At the same time, research quality has more dimensions than productivity and impact alone. As current bibliometric indicators are not covering various important quality dimensions, we here contribute to developing better indicators for those quality dimensions not yet addressed. One of the quality dimensions lacking valid indicators is an individual researcher’s independence. We propose indicators to measure different aspects of independence: two assessing whether a researcher has developed an own collaboration network and two others assessing the level of thematic independence. Taken together they form an independence indicator. We illustrate how these indicators distinguish between researchers that are equally productive and have a considerable impact. The independence indicator is a step forward in evaluating individual scholarly quality.

Suggested Citation

  • Peter van den Besselaar & Ulf Sandström, 2019. "Measuring researcher independence using bibliometric data: A proposal for a new performance indicator," PLOS ONE, Public Library of Science, vol. 14(3), pages 1-20, March.
    • Handle: RePEc:plo:pone00:0202712
    DOI: 10.1371/journal.pone.0202712
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0202712
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0202712&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0202712?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Rodrigo Costas & Thed N. van Leeuwen & María Bordons, 2010. "A bibliometric classificatory approach for the study and assessment of research performance at the individual level: The effects of age on productivity and impact," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 61(8), pages 1564-1581, August.
    2. Christine Wennerås & Agnes Wold, 1997. "Nepotism and sexism in peer-review," Nature, Nature, vol. 387(6631), pages 341-343, May.
    3. Wang, Jian & Veugelers, Reinhilde & Stephan, Paula, 2017. "Bias against novelty in science: A cautionary tale for users of bibliometric indicators," Research Policy, Elsevier, vol. 46(8), pages 1416-1436.
    4. Vieira, Elizabeth S. & Cabral, José A.S. & Gomes, José A.N.F., 2014. "How good is a model based on bibliometric indicators in predicting the final decisions made by peers?," Journal of Informetrics, Elsevier, vol. 8(2), pages 390-405.
    5. Ludo Waltman & Nees Eck, 2013. "A smart local moving algorithm for large-scale modularity-based community detection," The European Physical Journal B: Condensed Matter and Complex Systems, Springer;EDP Sciences, vol. 86(11), pages 1-14, November.
    6. Ali Gazni & Cassidy R. Sugimoto & Fereshteh Didegah, 2012. "Mapping world scientific collaboration: Authors, institutions, and countries," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 63(2), pages 323-335, February.
    7. Ali Gazni & Cassidy R. Sugimoto & Fereshteh Didegah, 2012. "Mapping world scientific collaboration: Authors, institutions, and countries," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(2), pages 323-335, February.
    8. Dorte Henriksen, 2016. "The rise in co-authorship in the social sciences (1980–2013)," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(2), pages 455-476, May.
    9. Philippe Mongeon & Elise Smith & Bruno Joyal & Vincent Larivière, 2017. "The rise of the middle author: Investigating collaboration and division of labor in biomedical research using partial alphabetical authorship," PLOS ONE, Public Library of Science, vol. 12(9), pages 1-14, September.
    10. Qi Wang, 2018. "A bibliometric model for identifying emerging research topics," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 69(2), pages 290-304, February.
    11. Johan Bollen & Herbert Van de Sompel & Aric Hagberg & Ryan Chute, 2009. "A Principal Component Analysis of 39 Scientific Impact Measures," PLOS ONE, Public Library of Science, vol. 4(6), pages 1-11, June.
    12. Rodrigo Costas & Thed N. van Leeuwen & María Bordons, 2010. "A bibliometric classificatory approach for the study and assessment of research performance at the individual level: The effects of age on productivity and impact," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(8), pages 1564-1581, August.
    13. Duncan Thomas & Maria Nedeva, 2012. "Characterizing researchers to study research funding agency impacts: The case of the European Research Council's Starting Grants," Research Evaluation, Oxford University Press, vol. 21(4), pages 257-269, September.
    14. Lorna Wildgaard & Jesper W. Schneider & Birger Larsen, 2014. "A review of the characteristics of 108 author-level bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 125-158, October.
    15. Sven E. Hug & Michael Ochsner & Hans-Dieter Daniel, 2013. "Criteria for assessing research quality in the humanities: a Delphi study among scholars of English literature, German literature and art history," Research Evaluation, Oxford University Press, vol. 22(5), pages 369-383, August.
    16. van den Besselaar, Peter & Heyman, Ulf & Sandström, Ulf, 2017. "Perverse effects of output-based research funding? Butler’s Australian case revisited," Journal of Informetrics, Elsevier, vol. 11(3), pages 905-918.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Shu-Hao Chang, 2024. "International Technology Market Hotspots and Development Trends from the Perspective of Inventor Mobility," Journal of the Knowledge Economy, Springer;Portland International Center for Management of Engineering and Technology (PICMET), vol. 15(1), pages 2361-2382, March.
    2. Gerald Schweiger & Adrian Barnett & Peter van den Besselaar & Lutz Bornmann & Andreas De Block & John P. A. Ioannidis & Ulf Sandstrom & Stijn Conix, 2024. "The Costs of Competition in Distributing Scarce Research Funds," Papers 2403.16934, arXiv.org.
    3. Katarina Rojko & Brankica Bratić & Borut Lužar, 2020. "The Bologna reform’s impacts on the scientific publication performance of Ph.D. graduates—the case of Slovenia," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(1), pages 329-356, July.
    4. Rojko, Katarina & Lužar, Borut, 2022. "Scientific performance across research disciplines: Trends and differences in the case of Slovenia," Journal of Informetrics, Elsevier, vol. 16(2).
    5. Bruno S. Frey & Anthony Gullo, 2020. "Sic transit gloria mundi: What remains of famous economists after their deaths?," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(1), pages 283-298, April.
    6. Mariana-Daniela González-Zamar & Emilio Abad-Segura & Eloy López-Meneses & José Gómez-Galán, 2020. "Managing ICT for Sustainable Education: Research Analysis in the Context of Higher Education," Sustainability, MDPI, vol. 12(19), pages 1-25, October.
    7. Duarte-Martínez, V. & Cobo, M.J. & López-Herrera, A.G., 2022. "Uncovering patterns in the supervision of Spanish theses: a comprehensive analysis," Journal of Informetrics, Elsevier, vol. 16(3).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Tóth, Tamás & Demeter, Márton & Csuhai, Sándor & Major, Zsolt Balázs, 2024. "When career-boosting is on the line: Equity and inequality in grant evaluation, productivity, and the educational backgrounds of Marie Skłodowska-Curie Actions individual fellows in social sciences an," Journal of Informetrics, Elsevier, vol. 18(2).
    2. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    3. Marek Kwiek, 2020. "Internationalists and locals: international research collaboration in a resource-poor system," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(1), pages 57-105, July.
    4. Xingchen Li & Qiang Wu & Yuanyuan Liu, 2017. "A quantitative analysis of researcher citation personal display considering disciplinary differences and influence factors," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 1093-1112, November.
    5. Jielan Ding & Zhesi Shen & Per Ahlgren & Tobias Jeppsson & David Minguillo & Johan Lyhagen, 2021. "The link between ethnic diversity and scientific impact: the mediating effect of novelty and audience diversity," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 7759-7810, September.
    6. Andersen, Jens Peter, 2017. "An empirical and theoretical critique of the Euclidean index," Journal of Informetrics, Elsevier, vol. 11(2), pages 455-465.
    7. Wagner, Caroline S. & Whetsell, Travis A. & Mukherjee, Satyam, 2019. "International research collaboration: Novelty, conventionality, and atypicality in knowledge recombination," Research Policy, Elsevier, vol. 48(5), pages 1260-1270.
    8. Kiran Kaur & Kwan Hoong Ng & Ray Kemp & Yin Yee Ong & Zaharah Ramly & Ai Peng Koh, 2019. "Knowledge generation in the wake of the Fukushima Daiichi nuclear power plant disaster," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(1), pages 149-169, April.
    9. Sixto-Costoya Andrea & Robinson-Garcia Nicolas & Leeuwen Thed & Costas Rodrigo, 2021. "Exploring the relevance of ORCID as a source of study of data sharing activities at the individual-level: a methodological discussion," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(8), pages 7149-7165, August.
    10. Vincent Larivière & Rodrigo Costas, 2016. "How Many Is Too Many? On the Relationship between Research Productivity and Impact," PLOS ONE, Public Library of Science, vol. 11(9), pages 1-10, September.
    11. Rodrigo Costas & Thomas Franssen, 2018. "Reflections around ‘the cautionary use’ of the h-index: response to Teixeira da Silva and Dobránszki," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(2), pages 1125-1130, May.
    12. António Osório & Lutz Bornmann, 2021. "On the disruptive power of small-teams research," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(1), pages 117-133, January.
    13. Vîiu, Gabriel-Alexandru, 2017. "Disaggregated research evaluation through median-based characteristic scores and scales: a comparison with the mean-based approach," Journal of Informetrics, Elsevier, vol. 11(3), pages 748-765.
    14. Mike Thelwall & Nabeil Maflahi, 2020. "Academic collaboration rates and citation associations vary substantially between countries and fields," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 71(8), pages 968-978, August.
    15. Xiaozan Lyu & Rodrigo Costas, 2021. "Studying the characteristics of scientific communities using individual-level bibliometrics: the case of Big Data research," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(8), pages 6965-6987, August.
    16. Panagopoulos, George & Tsatsaronis, George & Varlamis, Iraklis, 2017. "Detecting rising stars in dynamic collaborative networks," Journal of Informetrics, Elsevier, vol. 11(1), pages 198-222.
    17. Kaile Gong & Ying Cheng, 2022. "Patterns and impact of collaboration in China’s social sciences: cross-database comparisons between CSSCI and SSCI," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(10), pages 5947-5964, October.
    18. Abramo, Giovanni & D'Angelo, Ciriaco Andrea & Di Costa, Flavia, 2019. "Diversification versus specialization in scientific research: Which strategy pays off?," Technovation, Elsevier, vol. 82, pages 51-57.
    19. Cathelijn J F Waaijer & Benoît Macaluso & Cassidy R Sugimoto & Vincent Larivière, 2016. "Stability and Longevity in the Publication Careers of U.S. Doctorate Recipients," PLOS ONE, Public Library of Science, vol. 11(4), pages 1-15, April.
    20. Constance Poitras & Vincent Larivière, 2023. "Research mobility to the United States: a bibliometric analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(4), pages 2601-2614, April.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0202712. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.