IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v94y2013i3d10.1007_s11192-012-0775-9.html
   My bibliography  Save this article

Citation time window choice for research impact evaluation

Author

Listed:
  • Jian Wang

    (Institute for Research Information and Quality Assurance (iFQ))

Abstract

This paper aims to inform choice of citation time window for research evaluation, by answering three questions: (1) How accurate is it to use citation counts in short time windows to approximate total citations? (2) How does citation ageing vary by research fields, document types, publication months, and total citations? (3) Can field normalization improve the accuracy of using short citation time windows? We investigate the 31-year life time non-self-citation processes of all Thomson Reuters Web of Science journal papers published in 1980. The correlation between non-self-citation counts in each time window and total non-self-citations in all 31 years is calculated, and it is lower for more highly cited papers than less highly cited ones. There are significant differences in citation ageing between different research fields, document types, total citation counts, and publication months. However, the within group differences are more striking; many papers in the slowest ageing field may still age faster than many papers in the fastest ageing field. Furthermore, field normalization cannot improve the accuracy of using short citation time windows. Implications and recommendations for choosing adequate citation time windows are discussed.

Suggested Citation

  • Jian Wang, 2013. "Citation time window choice for research impact evaluation," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(3), pages 851-872, March.
    • Handle: RePEc:spr:scient:v:94:y:2013:i:3:d:10.1007_s11192-012-0775-9
    DOI: 10.1007/s11192-012-0775-9
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-012-0775-9
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-012-0775-9?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Alison Abbott, 2009. "Italy introduces performance-related funding," Nature, Nature, vol. 460(7255), pages 559-559, July.
    2. Abramo, Giovanni & Cicero, Tindaro & D’Angelo, Ciriaco Andrea, 2012. "A sensitivity analysis of researchers’ productivity rankings to the time of citation observation," Journal of Informetrics, Elsevier, vol. 6(2), pages 192-201.
    3. Loet Leydesdorff, 2008. "Caveats for the use of citation indicators in research and journal evaluations," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(2), pages 278-287, January.
    4. Rodrigo Costas & Thed N. van Leeuwen & Anthony F.J. van Raan, 2010. "Is scientific literature subject to a ‘Sell-By-Date’? A general methodology to analyze the ‘durability’ of scientific documents," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(2), pages 329-339, February.
    5. Wolfgang Glänzel & Balázs Schlemmer & Bart Thijs, 2003. "Better late than never? On the chance to become highly cited only beyond the standard bibliometric time horizon," Scientometrics, Springer;Akadémiai Kiadó, vol. 58(3), pages 571-586, November.
    6. Anthony F. J. van Raan, 2004. "Sleeping Beauties in science," Scientometrics, Springer;Akadémiai Kiadó, vol. 59(3), pages 467-472, March.
    7. Rodrigo Costas & Thed N. Leeuwen & Anthony F. J. Raan, 2011. "The “Mendel syndrome” in science: durability of scientific literature and its effects on bibliometric analysis of individual scientists," Scientometrics, Springer;Akadémiai Kiadó, vol. 89(1), pages 177-205, October.
    8. Loet Leydesdorff, 2009. "How are new citation‐based journal indicators adding to the bibliometric toolbox?," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(7), pages 1327-1336, July.
    9. Abramo, Giovanni & Cicero, Tindaro & D’Angelo, Ciriaco Andrea, 2012. "A sensitivity analysis of research institutions’ productivity rankings to the time of citation observation," Journal of Informetrics, Elsevier, vol. 6(2), pages 298-306.
    10. Glenn D. Walters, 2011. "The citation life cycle of articles published in 13 American Psychological Association journals: A 25-year longitudinal analysis," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 62(8), pages 1629-1636, August.
    11. Jonathan Adams, 2005. "Early citation counts correlate with accumulated impact," Scientometrics, Springer;Akadémiai Kiadó, vol. 63(3), pages 567-581, June.
    12. Juan D Rogers, 2010. "Citation analysis of nanotechnology at the field level: implications of R&D evaluation," Research Evaluation, Oxford University Press, vol. 19(4), pages 281-290, October.
    13. Dag W. Aksnes, 2003. "A macro study of self-citation," Scientometrics, Springer;Akadémiai Kiadó, vol. 56(2), pages 235-246, February.
    14. Wolfgang Glänzel & Koenraad Debackere & Bart Thijs & András Schubert, 2006. "A concise review on the role of author self-citations in information science, bibliometrics and science policy," Scientometrics, Springer;Akadémiai Kiadó, vol. 67(2), pages 263-277, May.
    15. Glenn D. Walters, 2011. "The citation life cycle of articles published in 13 American Psychological Association journals: A 25‐year longitudinal analysis," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 62(8), pages 1629-1636, August.
    16. Jonathan M. Levitt & Mike Thelwall, 2008. "Patterns of annual citation of highly cited articles and the prediction of their citation ranking: A comparison across subjects," Scientometrics, Springer;Akadémiai Kiadó, vol. 77(1), pages 41-60, October.
    17. Dag W Aksnes, 2003. "Characteristics of highly cited papers," Research Evaluation, Oxford University Press, vol. 12(3), pages 159-170, December.
    18. Abramo, Giovanni & Cicero, Tindaro & D’Angelo, Ciriaco Andrea, 2011. "Assessing the varying level of impact measurement accuracy as a function of the citation window length," Journal of Informetrics, Elsevier, vol. 5(4), pages 659-667.
    19. David A. King, 2004. "The scientific impact of nations," Nature, Nature, vol. 430(6997), pages 311-316, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    2. Bar-Ilan, Judit, 2008. "Informetrics at the beginning of the 21st century—A review," Journal of Informetrics, Elsevier, vol. 2(1), pages 1-52.
    3. Onodera, Natsuo, 2016. "Properties of an index of citation durability of an article," Journal of Informetrics, Elsevier, vol. 10(4), pages 981-1004.
    4. Chen, Huaqi & Ho, Yuh-Shan, 2015. "Highly cited articles in biomass research: A bibliometric analysis," Renewable and Sustainable Energy Reviews, Elsevier, vol. 49(C), pages 12-20.
    5. Lachance, Christian & Larivière, Vincent, 2014. "On the citation lifecycle of papers with delayed recognition," Journal of Informetrics, Elsevier, vol. 8(4), pages 863-872.
    6. Li, Jiang & Shi, Dongbo & Zhao, Star X. & Ye, Fred Y., 2014. "A study of the “heartbeat spectra” for “sleeping beauties”," Journal of Informetrics, Elsevier, vol. 8(3), pages 493-502.
    7. Rodrigo Costas & Thed N. Leeuwen & Anthony F. J. Raan, 2011. "The “Mendel syndrome” in science: durability of scientific literature and its effects on bibliometric analysis of individual scientists," Scientometrics, Springer;Akadémiai Kiadó, vol. 89(1), pages 177-205, October.
    8. Jiang Li, 2014. "Citation curves of “all-elements-sleeping-beauties”: “flash in the pan” first and then “delayed recognition”," Scientometrics, Springer;Akadémiai Kiadó, vol. 100(2), pages 595-601, August.
    9. Wang, Jian, 2016. "Knowledge creation in collaboration networks: Effects of tie configuration," Research Policy, Elsevier, vol. 45(1), pages 68-80.
    10. Jonathan M. Levitt & Mike Thelwall, 2009. "The most highly cited Library and Information Science articles: Interdisciplinarity, first authors and citation patterns," Scientometrics, Springer;Akadémiai Kiadó, vol. 78(1), pages 45-67, January.
    11. Jonathan M. Levitt & Mike Thelwall, 2008. "Patterns of annual citation of highly cited articles and the prediction of their citation ranking: A comparison across subjects," Scientometrics, Springer;Akadémiai Kiadó, vol. 77(1), pages 41-60, October.
    12. Marcel Clermont & Johanna Krolak & Dirk Tunger, 2021. "Does the citation period have any effect on the informative value of selected citation indicators in research evaluations?," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1019-1047, February.
    13. Guoliang Lyu & Ganwei Shi, 2019. "On an approach to boosting a journal’s citation potential," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(3), pages 1387-1409, September.
    14. Giovanni Abramo & Ciriaco Andrea D’Angelo & Tindaro Cicero, 2012. "What is the appropriate length of the publication period over which to assess research performance?," Scientometrics, Springer;Akadémiai Kiadó, vol. 93(3), pages 1005-1017, December.
    15. You Song & Fangling Situ & Hongjun Zhu & Jinzhi Lei, 2018. "To be the Prince to wake up Sleeping Beauty: the rediscovery of the delayed recognition studies," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(1), pages 9-24, October.
    16. Hui Fang, 2019. "A transition stage co-citation criterion for identifying the awakeners of sleeping beauty publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(1), pages 307-322, October.
    17. Aksnes, Dag W. & Rip, Arie, 2009. "Researchers' perceptions of citations," Research Policy, Elsevier, vol. 38(6), pages 895-905, July.
    18. Adil El Aichouchi & Philippe Gorry, 2018. "Delayed recognition of Judah Folkman’s hypothesis on tumor angiogenesis: when a Prince awakens a Sleeping Beauty by self-citation," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(1), pages 385-399, July.
    19. Lutz Bornmann & Werner Marx, 2014. "How to evaluate individual researchers working in the natural and life sciences meaningfully? A proposal of methods based on percentiles of citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(1), pages 487-509, January.
    20. Leo Egghe, 2007. "Probabilities for encountering genius, basic, ordinary or insignificant papers based on the cumulative nth citation distribution," Scientometrics, Springer;Akadémiai Kiadó, vol. 70(1), pages 167-181, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:94:y:2013:i:3:d:10.1007_s11192-012-0775-9. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.