IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v129y2024i10d10.1007_s11192-024-05028-w.html
   My bibliography  Save this article

Tracking changes in CNCI: the complementarity of standard, collaboration and fractional CNCI in understanding and evaluating research performance

Author

Listed:
  • Ross W. K. Potter

    (Clarivate)

  • Milovan Kovač

    (Clarivate)

  • Jonathan Adams

    (Clarivate
    King’s College London)

Abstract

Using a global set of ~ 300 institutions, standard, collaboration and fractional Category Normalised Citation Impact (CNCI) indicators are compared between 2009 and 2018 to demonstrate the complementarity of the three variants for research evaluation. Web of Science data show that Chinese institutions appear immune to the indicator used as CNCI changes, generally improvements, are similar for all three variants. Other regions tend to show greater increases in standard CNCI over collaboration CNCI, which in turn is greater than fractional CNCI; however, decreases in CNCI values, particularly in established research economies like North America and western Europe are not uncommon. These findings may highlight the differing extent to which the number of collaborating countries and institutions on papers affect each variant. Other factors affecting CNCI values may be citation practices and hiring of Highly Cited Researchers. Evaluating and comparing the performance of institutions is a main driver of policy, research and funding direction. Decision makers must understand all aspects of CNCI indicators, including the secondary factors illustrated here, by using a ‘profiles not metrics’ approach.

Suggested Citation

  • Ross W. K. Potter & Milovan Kovač & Jonathan Adams, 2024. "Tracking changes in CNCI: the complementarity of standard, collaboration and fractional CNCI in understanding and evaluating research performance," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(10), pages 6183-6196, October.
  • Handle: RePEc:spr:scient:v:129:y:2024:i:10:d:10.1007_s11192-024-05028-w
    DOI: 10.1007/s11192-024-05028-w
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-024-05028-w
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s11192-024-05028-w?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Diana Hicks & J Sylvan Katz, 1996. "Science policy for a highly collaborative science system," Science and Public Policy, Oxford University Press, vol. 23(1), pages 39-44, February.
    2. Adams, Jonathan, 2018. "Information and misinformation in bibliometric time-trend analysis," Journal of Informetrics, Elsevier, vol. 12(4), pages 1063-1071.
    3. Victoria Bakare & Grant Lewison, 2017. "Country over-citation ratios," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 1199-1207, November.
    4. Potter, Ross W.K. & Szomszor, Martin & Adams, Jonathan, 2020. "Interpreting CNCIs on a country-scale: The effect of domestic and international collaboration type," Journal of Informetrics, Elsevier, vol. 14(4).
    5. Leonardo Costa Ribeiro & Márcia Siqueira Rapini & Leandro Alves Silva & Eduardo Motta Albuquerque, 2018. "Growth patterns of the network of international collaboration in science," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(1), pages 159-179, January.
    6. Loet Leydesdorff & Ping Zhou, 2005. "Are the contributions of China and Korea upsetting the world system of science?," Scientometrics, Springer;Akadémiai Kiadó, vol. 63(3), pages 617-630, June.
    7. Christoph Bartneck & Servaas Kokkelmans, 2011. "Detecting h-index manipulation through self-citation analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(1), pages 85-98, April.
    8. Sivertsen, Gunnar & Rousseau, Ronald & Zhang, Lin, 2019. "Measuring scientific contributions with modified fractional counting," Journal of Informetrics, Elsevier, vol. 13(2), pages 679-694.
    9. Wagner, Caroline S. & Leydesdorff, Loet, 2005. "Network structure, self-organization, and the growth of international collaboration in science," Research Policy, Elsevier, vol. 34(10), pages 1608-1618, December.
    10. Katz, J. Sylvan & Martin, Ben R., 1997. "What is research collaboration?," Research Policy, Elsevier, vol. 26(1), pages 1-18, March.
    11. Loet Leydesdorff & Jung C. Shin, 2011. "How to evaluate universities in terms of their relative citation impacts: Fractional counting of citations and the normalization of differences among disciplines," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 62(6), pages 1146-1155, June.
    12. Michel Zitt & Henry Small, 2008. "Modifying the journal impact factor by fractional citation weighting: The audience factor," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(11), pages 1856-1860, September.
    13. Jonathan Adams, 2013. "The fourth age of research," Nature, Nature, vol. 497(7451), pages 557-560, May.
    14. Ludo Waltman & Nees Jan Eck, 2013. "Source normalized indicators of citation impact: an overview of different approaches and an empirical comparison," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(3), pages 699-716, September.
    15. Leo Egghe & Ronald Rousseau & Guido Van Hooydonk, 2000. "Methods for accrediting publications to authors or countries: Consequences for evaluation studies," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 51(2), pages 145-157.
    16. Ibrahim Shehatta & Abdullah M. Al-Rubaish, 2019. "Impact of country self-citations on bibliometric indicators and ranking of most productive countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(2), pages 775-791, August.
    17. Jonathan Adams & Karen Gurney & Stuart Marshall, 2007. "Profiling citation impact: A new methodology," Scientometrics, Springer;Akadémiai Kiadó, vol. 72(2), pages 325-344, August.
    18. Li Tang & Philip Shapira & Jan Youtie, 2015. "Is there a clubbing effect underlying Chinese research citation Increases?," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(9), pages 1923-1932, September.
    19. Wang, Xing & Zhang, Zhihui, 2020. "Improving the reliability of short-term citation impact indicators by taking into account the correlation between short- and long-term citation impact," Journal of Informetrics, Elsevier, vol. 14(2).
    20. Quentin Burrell & Ronald Rousseau, 1995. "Fractional counts for authorship attribution: A numerical study," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 46(2), pages 97-102, March.
    21. Wolfgang Glänzel & Cornelius de Lange, 2002. "A distributional approach to multinationality measures of international scientific collaboration," Scientometrics, Springer;Akadémiai Kiadó, vol. 54(1), pages 75-89, April.
    22. Waltman, Ludo & van Eck, Nees Jan, 2015. "Field-normalized citation impact indicators and the choice of an appropriate counting method," Journal of Informetrics, Elsevier, vol. 9(4), pages 872-894.
    23. Mahdi Khelfaoui & Julien Larrègue & Vincent Larivière & Yves Gingras, 2020. "Measuring national self-referencing patterns of major science producers," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(2), pages 979-996, May.
    24. Aksnes, Dag W. & Schneider, Jesper W. & Gunnarsson, Magnus, 2012. "Ranking national research systems by citation indicators. A comparative analysis using whole and fractionalised counting methods," Journal of Informetrics, Elsevier, vol. 6(1), pages 36-43.
    25. G. Van Hooydonk, 1997. "Fractional counting of multiauthored publications: Consequences for the impact of authors," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 48(10), pages 944-945, October.
    26. Lutz Bornmann & Richard Williams, 2020. "An evaluation of percentile measures of citation impact, and a proposal for making them better," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(2), pages 1457-1478, August.
    27. Ross W. K. Potter & Martin Szomszor & Jonathan Adams, 2022. "Comparing standard, collaboration and fractional CNCI at the institutional level: Consequences for performance evaluation," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 7435-7448, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ross W. K. Potter & Martin Szomszor & Jonathan Adams, 2022. "Comparing standard, collaboration and fractional CNCI at the institutional level: Consequences for performance evaluation," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 7435-7448, December.
    2. Potter, Ross W.K. & Szomszor, Martin & Adams, Jonathan, 2020. "Interpreting CNCIs on a country-scale: The effect of domestic and international collaboration type," Journal of Informetrics, Elsevier, vol. 14(4).
    3. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    4. Jeffrey Demaine, 2022. "Fractionalization of research impact reveals global trends in university collaboration," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(5), pages 2235-2247, May.
    5. Elizabeth S. Vieira, 2023. "The influence of research collaboration on citation impact: the countries in the European Innovation Scoreboard," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(6), pages 3555-3579, June.
    6. Martin Szomszor & David A. Pendlebury & Jonathan Adams, 2020. "How much is too much? The difference between research influence and self-citation excess," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(2), pages 1119-1147, May.
    7. Jonathan Adams & Jo Johnson & Jonathan Grant, 2022. "The rise of UK–China research collaboration: Trends, opportunities and challenges [The West Should Start Sending Its Scientists to China]," Science and Public Policy, Oxford University Press, vol. 49(1), pages 132-147.
    8. Bouyssou, Denis & Marchant, Thierry, 2016. "Ranking authors using fractional counting of citations: An axiomatic approach," Journal of Informetrics, Elsevier, vol. 10(1), pages 183-199.
    9. Rahman, Mohammad Tariqur & Regenstein, Joe Mac & Kassim, Noor Lide Abu & Haque, Nazmul, 2017. "The need to quantify authors’ relative intellectual contributions in a multi-author paper," Journal of Informetrics, Elsevier, vol. 11(1), pages 275-281.
    10. Fairclough, Ruth & Thelwall, Mike, 2015. "National research impact indicators from Mendeley readers," Journal of Informetrics, Elsevier, vol. 9(4), pages 845-859.
    11. Mingers, John & Leydesdorff, Loet, 2015. "A review of theory and practice in scientometrics," European Journal of Operational Research, Elsevier, vol. 246(1), pages 1-19.
    12. Wagner, Caroline S. & Whetsell, Travis A. & Mukherjee, Satyam, 2019. "International research collaboration: Novelty, conventionality, and atypicality in knowledge recombination," Research Policy, Elsevier, vol. 48(5), pages 1260-1270.
    13. Csomós, György, 2020. "Introducing recalibrated academic performance indicators in the evaluation of individuals’ research performance: A case study from Eastern Europe," Journal of Informetrics, Elsevier, vol. 14(4).
    14. Laurent R. Bergé, 2017. "Network proximity in the geography of research collaboration," Papers in Regional Science, Wiley Blackwell, vol. 96(4), pages 785-815, November.
    15. Svein Kyvik & Ingvild Reymert, 2017. "Research collaboration in groups and networks: differences across academic fields," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 951-967, November.
    16. Jo Royle & Louisa Coles & Dorothy Williams & Paul Evans, 2007. "Publishing in international journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 71(1), pages 59-86, April.
    17. Hans Pohl, 2024. "Using citation-based indicators to compare bilateral research collaborations," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(8), pages 4751-4770, August.
    18. Cristiano Varin & Manuela Cattelan & David Firth, 2016. "Statistical modelling of citation exchange between statistics journals," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 179(1), pages 1-63, January.
    19. Graf, Holger & Kalthaus, Martin, 2018. "International research networks: Determinants of country embeddedness," Research Policy, Elsevier, vol. 47(7), pages 1198-1214.
    20. Bar-Ilan, Judit, 2008. "Informetrics at the beginning of the 21st century—A review," Journal of Informetrics, Elsevier, vol. 2(1), pages 1-52.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:129:y:2024:i:10:d:10.1007_s11192-024-05028-w. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.