IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v6y2012i4p470-479.html
   My bibliography  Save this article

Revisiting the scaling of citations for research assessment

Author

Listed:
  • Abramo, Giovanni
  • Cicero, Tindaro
  • D’Angelo, Ciriaco Andrea

Abstract

Over the past decade, national research evaluation exercises, traditionally conducted using the peer review method, have begun opening to bibliometric indicators. The citations received by a publication are assumed as proxy for its quality, but they require standardization prior to use in comparative evaluation of organizations or individual scientists: the citation data must be standardized, due to the varying citation behavior across research fields. The objective of this paper is to compare the effectiveness of the different methods of normalizing citations, in order to provide useful indications to research assessment practitioners. Simulating a typical national research assessment exercise, he analysis is conducted for all subject categories in the hard sciences and is based on the Thomson Reuters Science Citation Index-Expanded®. Comparisons show that the citations average is the most effective scaling parameter, when the average is based only on the publications actually cited.

Suggested Citation

  • Abramo, Giovanni & Cicero, Tindaro & D’Angelo, Ciriaco Andrea, 2012. "Revisiting the scaling of citations for research assessment," Journal of Informetrics, Elsevier, vol. 6(4), pages 470-479.
    • Handle: RePEc:eee:infome:v:6:y:2012:i:4:p:470-479
    DOI: 10.1016/j.joi.2012.03.005
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1751157712000272
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.joi.2012.03.005?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Lundberg, Jonas, 2007. "Lifting the crown—citation z-score," Journal of Informetrics, Elsevier, vol. 1(2), pages 145-154.
    2. Vieira, E.S. & Gomes, J.A.N.F., 2010. "Citations to scientific articles: Its distribution and dependence on the article features," Journal of Informetrics, Elsevier, vol. 4(1), pages 1-13.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhihui Zhang & Ying Cheng & Nian Cai Liu, 2015. "Improving the normalization effect of mean-based method from the perspective of optimization: optimization-based linear methods and their performance," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 587-607, January.
    2. Schneider, Jesper W., 2013. "Caveats for using statistical significance tests in research assessments," Journal of Informetrics, Elsevier, vol. 7(1), pages 50-62.
    3. Sangwal, Keshra, 2014. "Distributions of citations of papers of individual authors publishing in different scientific disciplines: Application of Langmuir-type function," Journal of Informetrics, Elsevier, vol. 8(4), pages 972-984.
    4. Abramo, Giovanni & Cicero, Tindaro & D’Angelo, Ciriaco Andrea, 2012. "How important is choice of the scaling factor in standardizing citations?," Journal of Informetrics, Elsevier, vol. 6(4), pages 645-654.
    5. Vinkler, Péter, 2013. "Comparative rank assessment of journal articles," Journal of Informetrics, Elsevier, vol. 7(3), pages 712-717.
    6. Ma, Chao & Li, Yiwei & Guo, Feng & Si, Kao, 2019. "The citation trap: Papers published at year-end receive systematically fewer citations," Journal of Economic Behavior & Organization, Elsevier, vol. 166(C), pages 667-687.
    7. Bornmann, Lutz & Leydesdorff, Loet & Wang, Jian, 2014. "How to improve the prediction based on citation impact percentiles for years shortly after the publication date?," Journal of Informetrics, Elsevier, vol. 8(1), pages 175-180.
    8. Patrick Herron & Aashish Mehta & Cong Cao & Timothy Lenoir, 2016. "Research diversification and impact: the case of national nanoscience development," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 629-659, November.
    9. Kong, Ling & Wang, Dongbo, 2020. "Comparison of citations and attention of cover and non-cover papers," Journal of Informetrics, Elsevier, vol. 14(4).
    10. Mehdi Rhaiem & Nabil Amara, 2020. "Determinants of research efficiency in Canadian business schools: evidence from scholar-level data," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 53-99, October.
    11. Bárbara S. Lancho-Barrantes & Vicente P. Guerrero-Bote & Félix Moya-Anegón, 2010. "The iceberg hypothesis revisited," Scientometrics, Springer;Akadémiai Kiadó, vol. 85(2), pages 443-461, November.
    12. Mingyang Wang & Zhenyu Wang & Guangsheng Chen, 2019. "Which can better predict the future success of articles? Bibliometric indices or alternative metrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(3), pages 1575-1595, June.
    13. Abramo, Giovanni & Cicero, Tindaro & D’Angelo, Ciriaco Andrea, 2012. "The dispersion of research performance within and between universities as a potential indicator of the competitive intensity in higher education systems," Journal of Informetrics, Elsevier, vol. 6(2), pages 155-168.
    14. Sabrina Petersohn & Thomas Heinze, 2018. "Professionalization of bibliometric research assessment. Insights from the history of the Leiden Centre for Science and Technology Studies (CWTS)," Science and Public Policy, Oxford University Press, vol. 45(4), pages 565-578.
    15. João Claro & Carlos A. V. Costa, 2011. "A made-to-measure indicator for cross-disciplinary bibliometric ranking of researchers performance," Scientometrics, Springer;Akadémiai Kiadó, vol. 86(1), pages 113-123, January.
    16. Jens Peter Andersen & Björn Hammarfelt, 2011. "Price revisited: on the growth of dissertations in eight research fields," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(2), pages 371-383, August.
    17. Chi, Yuxue & Tang, Xianyi & Liu, Yijun, 2022. "Exploring the “awakening effect” in knowledge diffusion: a case study of publications in the library and information science domain," Journal of Informetrics, Elsevier, vol. 16(4).
    18. Giovanni Abramo & Ciriaco Andrea D’Angelo & Flavia Di Costa, 2011. "A national-scale cross-time analysis of university research performance," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(2), pages 399-413, May.
    19. Didegah, Fereshteh & Thelwall, Mike, 2013. "Which factors help authors produce the highest impact research? Collaboration, journal and document properties," Journal of Informetrics, Elsevier, vol. 7(4), pages 861-873.
    20. José Satsumi López-Morales & Héctor Francisco Salazar-Núñez & Claudia Guadalupe Zarrabal-Gutiérrez, 2022. "The impact of qualitative methods on article citation: an international business research perspective," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(6), pages 3225-3236, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:6:y:2012:i:4:p:470-479. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joi .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.