IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v11y2017i3p672-684.html
   My bibliography  Save this article

A categorization of arguments for counting methods for publication and citation indicators

Author

Listed:
  • Gauffriau, Marianne

Abstract

Most publication and citation indicators are based on datasets with multi-authored publications and thus a change in counting method will often change the value of an indicator. Therefore it is important to know why a specific counting method has been applied. I have identified arguments for counting methods in a sample of 32 bibliometric studies published in 2016 and compared the result with discussions of arguments for counting methods in three older studies. Based on the underlying logics of the arguments I have arranged the arguments in four groups. Group 1 focuses on arguments related to what an indicator measures, Group 2 on the additivity of a counting method, Group 3 on pragmatic reasons for the choice of counting method, and Group 4 on an indicator’s influence on the research community or how it is perceived by researchers. This categorization can be used to describe and discuss how bibliometric studies with publication and citation indicators argue for counting

Suggested Citation

  • Gauffriau, Marianne, 2017. "A categorization of arguments for counting methods for publication and citation indicators," Journal of Informetrics, Elsevier, vol. 11(3), pages 672-684.
    • Handle: RePEc:eee:infome:v:11:y:2017:i:3:p:672-684
    DOI: 10.1016/j.joi.2017.05.009
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1751157716302760
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.joi.2017.05.009?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Jörg Neufeld, 2016. "Determining effects of individual research grants on publication output and impact: The case of the Emmy Noether Programme (German Research Foundation)," Research Evaluation, Oxford University Press, vol. 25(1), pages 50-61.
    2. Marianne Gauffriau & Peder Olesen Larsen & Isabelle Maye & Anne Roulin-Perriard & Markus Ins, 2007. "Publication, cooperation and productivity measures in scientific research," Scientometrics, Springer;Akadémiai Kiadó, vol. 73(2), pages 175-214, November.
    3. Daiji Kawaguchi & Ayako Kondo & Keiji Saito, 2016. "Researchers’ career transitions over the life cycle," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1435-1454, December.
    4. Koski, Timo & Sandström, Erik & Sandström, Ulf, 2016. "Towards field-adjusted production: Estimating research productivity from a zero-truncated distribution," Journal of Informetrics, Elsevier, vol. 10(4), pages 1143-1152.
    5. Finn Valentin & Maria Theresa Norn & Lars Alkaersig, 2016. "Orientations and outcome of interdisciplinary research: the case of research behaviour in translational medical science," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(1), pages 67-90, January.
    6. Abramo, Giovanni & D’Angelo, Ciriaco Andrea & Rosati, Francesco, 2016. "A methodology to measure the effectiveness of academic recruitment and turnover," Journal of Informetrics, Elsevier, vol. 10(1), pages 31-42.
    7. Peter van den Besselaar & Ulf Sandström, 2016. "Gender differences in research performance and its impact on careers: a longitudinal case study," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(1), pages 143-162, January.
    8. Noriyuki Morichika & Sotaro Shibayama, 2016. "Use of dissertation data in science policy research," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(1), pages 221-241, July.
    9. Ana Ramos & Cláudia S. Sarrico, 2016. "Past performance does not guarantee future results: lessons from the evaluation of research units in Portugal," Research Evaluation, Oxford University Press, vol. 25(1), pages 94-106.
    10. Marianne Gauffriau & Peder Olesen Larsen & Isabelle Maye & Anne Roulin-Perriard & Markus Ins, 2008. "Comparisons of results of publication counting using different methods," Scientometrics, Springer;Akadémiai Kiadó, vol. 77(1), pages 147-176, October.
    11. Cimini, Giulio & Zaccaria, Andrea & Gabrielli, Andrea, 2016. "Investigating the interplay between fundamentals of national research systems: Performance, investments and international collaborations," Journal of Informetrics, Elsevier, vol. 10(1), pages 200-211.
    12. Peder Olesen Larsen, 2008. "The state of the art in publication counting," Scientometrics, Springer;Akadémiai Kiadó, vol. 77(2), pages 235-251, November.
    13. Elias Sanz-Casado & Carlos García-Zorita & Ronald Rousseau, 2016. "Using h-cores to study the most-cited articles of the twenty-first century," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(1), pages 243-261, July.
    14. Grant Lewison & Philip Roe & Richard Webber & Richard Sullivan, 2016. "Lung cancer researchers, 2008–2013: their sex and ethnicity," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(1), pages 105-117, January.
    15. Kahn, Shulamit & MacGarvie, Megan, 2016. "Do return requirements increase international knowledge diffusion? Evidence from the Fulbright program," Research Policy, Elsevier, vol. 45(6), pages 1304-1322.
    16. Krzysztof Klincewicz, 2016. "The emergent dynamics of a technological research topic: the case of graphene," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(1), pages 319-345, January.
    17. Waltman, Ludo & van Eck, Nees Jan, 2015. "Field-normalized citation impact indicators and the choice of an appropriate counting method," Journal of Informetrics, Elsevier, vol. 9(4), pages 872-894.
    18. B. Pritychenko, 2016. "Fractional authorship in nuclear physics," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(1), pages 461-468, January.
    19. Sonia R. Zanotto & Cristina Haeffner & Jorge A. Guimarães, 2016. "Unbalanced international collaboration affects adversely the usefulness of countries’ scientific output as well as their technological and social impact," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1789-1814, December.
    20. Yuret, Tolga, 2016. "Interfield equality: Journals versus researchers," Journal of Informetrics, Elsevier, vol. 10(4), pages 1196-1206.
    21. Marco Frittelli & Loriano Mancini & Ilaria Peri, 2016. "Scientific research measures," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 67(12), pages 3051-3063, December.
    22. Tuomas Höylä & Christoph Bartneck & Timo Tiihonen, 2016. "The consequences of competition: simulating the effects of research grant allocation strategies," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(1), pages 263-288, July.
    23. Aksnes, Dag W. & Schneider, Jesper W. & Gunnarsson, Magnus, 2012. "Ranking national research systems by citation indicators. A comparative analysis using whole and fractionalised counting methods," Journal of Informetrics, Elsevier, vol. 6(1), pages 36-43.
    24. Torger Möller & Marion Schmidt & Stefan Hornbostel, 2016. "Assessing the effects of the German Excellence Initiative with bibliometric methods," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 2217-2239, December.
    25. Thelwall, Mike & Sud, Pardeep, 2016. "National, disciplinary and temporal variations in the extent to which articles with more authors have more impact: Evidence from a geometric field normalised citation indicator," Journal of Informetrics, Elsevier, vol. 10(1), pages 48-61.
    26. Wildgaard, Lorna, 2016. "A critical cluster analysis of 44 indicators of author-level performance," Journal of Informetrics, Elsevier, vol. 10(4), pages 1055-1078.
    27. Giovanni Abramo & Ciriaco Andrea D’Angelo, 2016. "Refrain from adopting the combination of citation and journal metrics to grade publications, as used in the Italian national research assessment exercise (VQR 2011–2014)," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 2053-2065, December.
    28. Fredrik Niclas Piro & Gunnar Sivertsen, 2016. "How can differences in international university rankings be explained?," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 2263-2278, December.
    29. Abramo, Giovanni & Cicero, Tindaro & D’Angelo, Ciriaco Andrea, 2013. "Individual research performance: A proposal for comparing apples to oranges," Journal of Informetrics, Elsevier, vol. 7(2), pages 528-539.
    30. Grant Lewison & Sameer Kumar & Chan-Yuan Wong & Philip Roe & Richard Webber, 2016. "The contribution of ethnic groups to Malaysian scientific output, 1982–2014, and the effects of the new economic policy," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1877-1893, December.
    31. Bornmann, Lutz & Stefaner, Moritz & de Moya Anegón, Felix & Mutz, Rüdiger, 2016. "Excellence networks in science: A Web-based application based on Bayesian multilevel logistic regression (BMLR) for the identification of institutions collaborating successfully," Journal of Informetrics, Elsevier, vol. 10(1), pages 312-327.
    32. Clemens B. Fell & Cornelius J. König, 2016. "Is there a gender difference in scientific collaboration? A scientometric examination of co-authorships among industrial–organizational psychologists," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(1), pages 113-141, July.
    33. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Csomós, György, 2020. "Introducing recalibrated academic performance indicators in the evaluation of individuals’ research performance: A case study from Eastern Europe," Journal of Informetrics, Elsevier, vol. 14(4).
    2. Wolfgang G. Stock & Isabelle Dorsch & Gerhard Reichmann & Christian Schlögl, 2023. "Labor productivity, labor impact, and co-authorship of research institutions: publications and citations per full-time equivalents," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(1), pages 363-377, January.
    3. Nguyen, Minh-Hoang & Huyen, Nguyen Thanh Thanh & Pham, Thanh-Hang & Yen, Nguyen Thi Quynh & Vuong, Quan-Hoang, 2020. "On the 50-year research landscape of entrepreneurial finance: A sign of Western ideological homogeneity?," OSF Preprints qf62s, Center for Open Science.
    4. Vladimir Batagelj, 2020. "On fractional approach to analysis of linked networks," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(2), pages 621-633, May.
    5. Zaida Chinchilla-Rodríguez & Yi Bu & Nicolás Robinson-García & Cassidy R. Sugimoto, 2021. "An empirical review of the different variants of the probabilistic affinity index as applied to scientific collaboration," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1775-1795, February.
    6. Hou, Lei & Pan, Yueling & Zhu, Jonathan J.H., 2021. "Impact of scientific, economic, geopolitical, and cultural factors on international research collaboration," Journal of Informetrics, Elsevier, vol. 15(3).
    7. Korytkowski, Przemyslaw & Kulczycki, Emanuel, 2019. "Publication counting methods for a national research evaluation exercise," Journal of Informetrics, Elsevier, vol. 13(3), pages 804-816.
    8. Zaida Chinchilla-Rodríguez & Cassidy R Sugimoto & Vincent Larivière, 2019. "Follow the leader: On the relationship between leadership and scholarly impact in international collaborations," PLOS ONE, Public Library of Science, vol. 14(6), pages 1-18, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Perianes-Rodriguez, Antonio & Waltman, Ludo & van Eck, Nees Jan, 2016. "Constructing bibliometric networks: A comparison between full and fractional counting," Journal of Informetrics, Elsevier, vol. 10(4), pages 1178-1195.
    2. Jesper W. Schneider & Thed Leeuwen & Martijn Visser & Kaare Aagaard, 2019. "Examining national citation impact by comparing developments in a fixed and a dynamic journal set," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(2), pages 973-985, May.
    3. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    4. Korytkowski, Przemyslaw & Kulczycki, Emanuel, 2019. "Publication counting methods for a national research evaluation exercise," Journal of Informetrics, Elsevier, vol. 13(3), pages 804-816.
    5. Waltman, Ludo & van Eck, Nees Jan, 2015. "Field-normalized citation impact indicators and the choice of an appropriate counting method," Journal of Informetrics, Elsevier, vol. 9(4), pages 872-894.
    6. Zharova, Alona & Härdle, Wolfgang Karl & Lessmann, Stefan, 2023. "Data-driven support for policy and decision-making in university research management: A case study from Germany," European Journal of Operational Research, Elsevier, vol. 308(1), pages 353-368.
    7. Wu, Jiang & Ou, Guiyan & Liu, Xiaohui & Dong, Ke, 2022. "How does academic education background affect top researchers’ performance? Evidence from the field of artificial intelligence," Journal of Informetrics, Elsevier, vol. 16(2).
    8. Jeffrey Demaine, 2022. "Fractionalization of research impact reveals global trends in university collaboration," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(5), pages 2235-2247, May.
    9. Rahman, Mohammad Tariqur & Regenstein, Joe Mac & Kassim, Noor Lide Abu & Haque, Nazmul, 2017. "The need to quantify authors’ relative intellectual contributions in a multi-author paper," Journal of Informetrics, Elsevier, vol. 11(1), pages 275-281.
    10. Patelli, Aurelio & Napolitano, Lorenzo & Cimini, Giulio & Gabrielli, Andrea, 2023. "Geography of science: Competitiveness and inequality," Journal of Informetrics, Elsevier, vol. 17(1).
    11. Tanel Hirv, 2022. "The interplay of the size of the research system, ways of collaboration, level, and method of funding in determining bibliometric outputs," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(3), pages 1295-1316, March.
    12. Hagen, Nils T., 2014. "Counting and comparing publication output with and without equalizing and inflationary bias," Journal of Informetrics, Elsevier, vol. 8(2), pages 310-317.
    13. Zharova, Alona & Härdle, Wolfgang Karl & Lessmann, Stefan, 2017. "Is scientific performance a function of funds?," SFB 649 Discussion Papers 2017-028, Humboldt University Berlin, Collaborative Research Center 649: Economic Risk.
    14. Abramo, Giovanni & D'Angelo, Ciriaco Andrea & Di Costa, Flavia, 2022. "Revealing the scientific comparative advantage of nations: Common and distinctive features," Journal of Informetrics, Elsevier, vol. 16(1).
    15. Alona Zharova & Wolfgang K. Härdle & Stefan Lessmann, 2017. "Is Scientific Performance a Function of Funds?," SFB 649 Discussion Papers SFB649DP2017-028, Sonderforschungsbereich 649, Humboldt University, Berlin, Germany.
    16. Sandro Tarkhan-Mouravi, 2020. "Traditional indicators inflate some countries’ scientific impact over 10 times," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(1), pages 337-356, April.
    17. Ross W. K. Potter & Martin Szomszor & Jonathan Adams, 2022. "Comparing standard, collaboration and fractional CNCI at the institutional level: Consequences for performance evaluation," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 7435-7448, December.
    18. Matveeva, Nataliya & Sterligov, Ivan & Yudkevich, Maria, 2021. "The effect of Russian University Excellence Initiative on publications and collaboration patterns," Journal of Informetrics, Elsevier, vol. 15(1).
    19. Wolfgang G. Stock & Isabelle Dorsch & Gerhard Reichmann & Christian Schlögl, 2023. "Labor productivity, labor impact, and co-authorship of research institutions: publications and citations per full-time equivalents," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(1), pages 363-377, January.
    20. Potter, Ross W.K. & Szomszor, Martin & Adams, Jonathan, 2020. "Interpreting CNCIs on a country-scale: The effect of domestic and international collaboration type," Journal of Informetrics, Elsevier, vol. 14(4).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:11:y:2017:i:3:p:672-684. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joi .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.