IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v6y2012i4p645-654.html
   My bibliography  Save this article

How important is choice of the scaling factor in standardizing citations?

Author

Listed:
  • Abramo, Giovanni
  • Cicero, Tindaro
  • D’Angelo, Ciriaco Andrea

Abstract

Because of the variations in citation behavior across research fields, appropriate standardization must be applied as part of any bibliometric analysis of the productivity of individual scientists and research organizations. Such standardization involves scaling by some factor that characterizes the distribution of the citations of articles from the same year and subject category. In this work we conduct an analysis of the sensitivity of researchers’ productivity rankings to the scaling factor chosen to standardize their citations. To do this we first prepare the productivity rankings for all researchers (more than 30,000) operating in the hard sciences in Italy, over the period 2004–2008. We then measure the shifts in rankings caused by adopting scaling factors other than the particular factor that seems more effective for comparing the impact of publications in different fields: the citation average of the distribution of cited-only publications.

Suggested Citation

  • Abramo, Giovanni & Cicero, Tindaro & D’Angelo, Ciriaco Andrea, 2012. "How important is choice of the scaling factor in standardizing citations?," Journal of Informetrics, Elsevier, vol. 6(4), pages 645-654.
    • Handle: RePEc:eee:infome:v:6:y:2012:i:4:p:645-654
    DOI: 10.1016/j.joi.2012.07.002
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1751157712000533
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.joi.2012.07.002?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Pedro Albarrán & Juan A. Crespo & Ignacio Ortuño & Javier Ruiz-Castillo, 2011. "The skewness of science in 219 sub-fields and a number of aggregates," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(2), pages 385-397, August.
    2. H. P. F. Peters & A. F. J. van Raan, 1994. "On determinants of citation scores: A case study in chemical engineering," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 45(1), pages 39-49, January.
    3. Filippo Radicchi & Claudio Castellano, 2012. "A Reverse Engineering Approach to the Suppression of Citation Biases Reveals Universal Properties of Citation Distributions," PLOS ONE, Public Library of Science, vol. 7(3), pages 1-9, March.
    4. Loet Leydesdorff & Lutz Bornmann & Rüdiger Mutz & Tobias Opthof, 2011. "Turning the tables on citation analysis one more time: Principles for comparing sets of documents," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 62(7), pages 1370-1381, July.
    5. Lundberg, Jonas, 2007. "Lifting the crown—citation z-score," Journal of Informetrics, Elsevier, vol. 1(2), pages 145-154.
    6. S. Redner, 1998. "How popular is your paper? An empirical study of the citation distribution," The European Physical Journal B: Condensed Matter and Complex Systems, Springer;EDP Sciences, vol. 4(2), pages 131-134, July.
    7. Michael J. Stringer & Marta Sales-Pardo & Luís A. Nunes Amaral, 2010. "Statistical validation of a global model for the distribution of the ultimate number of citations accrued by papers published in a scientific journal," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(7), pages 1377-1385, July.
    8. Vieira, E.S. & Gomes, J.A.N.F., 2010. "Citations to scientific articles: Its distribution and dependence on the article features," Journal of Informetrics, Elsevier, vol. 4(1), pages 1-13.
    9. Ludo Waltman & Nees Jan van Eck & Anthony F. J. van Raan, 2012. "Universality of citation distributions revisited," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(1), pages 72-77, January.
    10. Ludo Waltman & Nees Jan van Eck & Anthony F. J. van Raan, 2012. "Universality of citation distributions revisited," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 63(1), pages 72-77, January.
    11. Abramo, Giovanni & Cicero, Tindaro & D’Angelo, Ciriaco Andrea, 2012. "Revisiting the scaling of citations for research assessment," Journal of Informetrics, Elsevier, vol. 6(4), pages 470-479.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Li, Yunrong & Radicchi, Filippo & Castellano, Claudio & Ruiz-Castillo, Javier, 2013. "Quantitative evaluation of alternative field normalization procedures," Journal of Informetrics, Elsevier, vol. 7(3), pages 746-755.
    2. Wu, Jiang, 2013. "Investigating the universal distributions of normalized indicators and developing field-independent index," Journal of Informetrics, Elsevier, vol. 7(1), pages 63-71.
    3. Abramo, Giovanni & D’Angelo, Ciriaco Andrea, 2017. "Does your surname affect the citability of your publications?," Journal of Informetrics, Elsevier, vol. 11(1), pages 121-127.
    4. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    5. Zhihui Zhang & Ying Cheng & Nian Cai Liu, 2014. "Comparison of the effect of mean-based method and z-score for field normalization of citations at the level of Web of Science subject categories," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(3), pages 1679-1693, December.
    6. Marcello D’Agostino & Valentino Dardanoni & Roberto Ghiselli Ricci, 2017. "How to standardize (if you must)," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 825-843, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhihui Zhang & Ying Cheng & Nian Cai Liu, 2015. "Improving the normalization effect of mean-based method from the perspective of optimization: optimization-based linear methods and their performance," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 587-607, January.
    2. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    3. Bouyssou, Denis & Marchant, Thierry, 2016. "Ranking authors using fractional counting of citations: An axiomatic approach," Journal of Informetrics, Elsevier, vol. 10(1), pages 183-199.
    4. Thelwall, Mike & Wilson, Paul, 2014. "Distributions for cited articles from individual subjects and years," Journal of Informetrics, Elsevier, vol. 8(4), pages 824-839.
    5. Zhihui Zhang & Ying Cheng & Nian Cai Liu, 2014. "Comparison of the effect of mean-based method and z-score for field normalization of citations at the level of Web of Science subject categories," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(3), pages 1679-1693, December.
    6. Dunaiski, Marcel & Geldenhuys, Jaco & Visser, Willem, 2019. "On the interplay between normalisation, bias, and performance of paper impact metrics," Journal of Informetrics, Elsevier, vol. 13(1), pages 270-290.
    7. Vîiu, Gabriel-Alexandru, 2018. "The lognormal distribution explains the remarkable pattern documented by characteristic scores and scales in scientometrics," Journal of Informetrics, Elsevier, vol. 12(2), pages 401-415.
    8. Thelwall, Mike, 2016. "Are the discretised lognormal and hooked power law distributions plausible for citation data?," Journal of Informetrics, Elsevier, vol. 10(2), pages 454-470.
    9. T. S. Evans & N. Hopkins & B. S. Kaube, 2012. "Universality of performance indicators based on citation and reference counts," Scientometrics, Springer;Akadémiai Kiadó, vol. 93(2), pages 473-495, November.
    10. Giancarlo Ruocco & Cinzia Daraio, 2013. "An empirical approach to compare the performance of heterogeneous academic fields," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(3), pages 601-625, December.
    11. Ruiz-Castillo, Javier & Costas, Rodrigo, 2018. "Individual and field citation distributions in 29 broad scientific fields," Journal of Informetrics, Elsevier, vol. 12(3), pages 868-892.
    12. Ruiz-Castillo, Javier & Waltman, Ludo, 2015. "Field-normalized citation impact indicators using algorithmically constructed classification systems of science," Journal of Informetrics, Elsevier, vol. 9(1), pages 102-117.
    13. S. R. Goldberg & H. Anthony & T. S. Evans, 2015. "Modelling citation networks," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(3), pages 1577-1604, December.
    14. Ruiz-Castillo, Javier & Costas, Rodrigo, 2014. "The skewness of scientific productivity," Journal of Informetrics, Elsevier, vol. 8(4), pages 917-934.
    15. Wang, Xing & Zhang, Zhihui, 2020. "Improving the reliability of short-term citation impact indicators by taking into account the correlation between short- and long-term citation impact," Journal of Informetrics, Elsevier, vol. 14(2).
    16. Waltman, Ludo & van Eck, Nees Jan, 2013. "A systematic empirical comparison of different approaches for normalizing citation impact indicators," Journal of Informetrics, Elsevier, vol. 7(4), pages 833-849.
    17. Sangwal, Keshra, 2014. "Distributions of citations of papers of individual authors publishing in different scientific disciplines: Application of Langmuir-type function," Journal of Informetrics, Elsevier, vol. 8(4), pages 972-984.
    18. Vaccario, Giacomo & Medo, Matúš & Wider, Nicolas & Mariani, Manuel Sebastian, 2017. "Quantifying and suppressing ranking bias in a large citation network," Journal of Informetrics, Elsevier, vol. 11(3), pages 766-782.
    19. Javier Ruiz-Castillo, 2013. "The role of statistics in establishing the similarity of citation distributions in a static and a dynamic context," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(1), pages 173-181, July.
    20. Kaur, Jasleen & Ferrara, Emilio & Menczer, Filippo & Flammini, Alessandro & Radicchi, Filippo, 2015. "Quality versus quantity in scientific impact," Journal of Informetrics, Elsevier, vol. 9(4), pages 800-808.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:6:y:2012:i:4:p:645-654. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joi .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.