IDEAS home Printed from https://ideas.repec.org/a/gam/jsusta/v13y2021i17p9515-d620743.html
   My bibliography  Save this article

Have Academics’ Citation Patterns Changed in Response to the Rise of World University Rankings? A Test Using First-Citation Speeds

Author

Listed:
  • Soo Jeung Lee

    (Department of Education, Sejong University, Seoul 050060, Korea)

  • Christian Schneijderberg

    (International Centre for Higher Education Research (INCHER), University of Kassel, D-34109 Kassel, Germany)

  • Yangson Kim

    (Research Institute for Higher Education (RIHE), Hiroshima University, Higashi-Hiroshima 739-8512, Japan)

  • Isabel Steinhardt

    (International Centre for Higher Education Research (INCHER), University of Kassel, D-34109 Kassel, Germany)

Abstract

Academics may actively respond to the expectations of the academic status market, which have largely been shaped by the World University Rankings (WURs). This study empirically examines how academics’ citation patterns have changed in response to the rise of an “evaluation environment” in academia. We regard the WURs to be a macro-level trigger for cementing a bibliometric-based evaluation environment in academia. Our analyses of citation patterns in papers published in two higher education journals explicitly considered three distinct periods: the pre-WURs (1990–2003), the period of WURs implementation (2004–2010), and the period of adaption to WURs (2011–2017). We applied the nonparametric Kaplan–Meier method to compare first-citation speeds of papers published across the three periods. We found that not only has first-citation speed become faster, but first-citation probability has also increased following the emergence of the WURs. Applying Cox proportional hazard models to first-citation probabilities, we identified journal impact factors and third-party funding as factors influencing first-citation probability, while other author- and paper-related factors showed limited effects. We also found that the general effects of different factors on first-citation speeds have changed with the emergence of the WURs. The findings expand our understanding of the citation patterns of academics in the rise of WURs and provide practical grounds for research policy as well as higher education policy.

Suggested Citation

  • Soo Jeung Lee & Christian Schneijderberg & Yangson Kim & Isabel Steinhardt, 2021. "Have Academics’ Citation Patterns Changed in Response to the Rise of World University Rankings? A Test Using First-Citation Speeds," Sustainability, MDPI, vol. 13(17), pages 1-19, August.
  • Handle: RePEc:gam:jsusta:v:13:y:2021:i:17:p:9515-:d:620743
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2071-1050/13/17/9515/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/2071-1050/13/17/9515/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Jeppe Nicolaisen & Tove Faber Frandsen, 2019. "Zero impact: a large-scale study of uncitedness," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(2), pages 1227-1254, May.
    2. Xin Gu & Karen Blackmore, 2017. "Quantitative study on Australian academic science," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 1009-1035, November.
    3. Jochen Gläser & Grit Laudel, 2005. "Advantages and dangers of ‘remote’ peer evaluation," Research Evaluation, Oxford University Press, vol. 14(3), pages 186-198, December.
    4. Rodrigo Costas & Thed N. Leeuwen & María Bordons, 2012. "Referencing patterns of individual researchers: Do top scientists rely on more extensive information sources?," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(12), pages 2433-2450, December.
    5. Egghe, L. & Bornmann, L. & Guns, R., 2011. "A proposal for a First-Citation-Speed-Index," Journal of Informetrics, Elsevier, vol. 5(1), pages 181-186.
    6. Butler, Linda, 2003. "Explaining Australia's increased share of ISI publications--the effects of a funding formula based on publication counts," Research Policy, Elsevier, vol. 32(1), pages 143-155, January.
    7. Peter Woelert & Lachlan McKenzie, 2018. "Follow the money? How Australian universities replicate national performance-based funding mechanisms," Research Evaluation, Oxford University Press, vol. 27(3), pages 184-195.
    8. Leo Egghe, 2000. "A Heuristic Study of the First-Citation Distribution," Scientometrics, Springer;Akadémiai Kiadó, vol. 48(3), pages 345-359, July.
    9. Lutz Bornmann & Rüdiger Mutz, 2015. "Growth rates of modern science: A bibliometric analysis based on the number of publications and cited references," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(11), pages 2215-2222, November.
    10. Linda Butler, 2003. "Modifying publication practices in response to funding formulas," Research Evaluation, Oxford University Press, vol. 12(1), pages 39-46, April.
    11. Henk F. Moed, 2017. "A critical comparative analysis of five world university rankings," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(2), pages 967-990, February.
    12. Anthony F. J. van Raan, 2004. "Sleeping Beauties in science," Scientometrics, Springer;Akadémiai Kiadó, vol. 59(3), pages 467-472, March.
    13. Rodrigo Costas & Thed N. van Leeuwen & María Bordons, 2012. "Referencing patterns of individual researchers: Do top scientists rely on more extensive information sources?," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 63(12), pages 2433-2450, December.
    14. Vladimir Pislyakov & Elena Shukshina, 2014. "Measuring excellence in Russia: Highly cited papers, leading institutions, patterns of national and international collaboration," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(11), pages 2321-2330, November.
    15. Laudel, Grit & Gläser, Jochen, 2014. "Beyond breakthrough research: Epistemic properties of research and their consequences for research funding," Research Policy, Elsevier, vol. 43(7), pages 1204-1216.
    16. Jan Youtie, 2014. "The use of citation speed to understand the effects of a multi-institutional science center," Scientometrics, Springer;Akadémiai Kiadó, vol. 100(3), pages 613-621, September.
    17. Star X. Zhao & Wen Lou & Alice M. Tan & Shuang Yu, 2018. "Do funded papers attract more usage?," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 153-168, April.
    18. Wolfgang Glänzel & Ronald Rousseau & Lin Zhang, 2012. "A visual representation of relative first‐citation times," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 63(7), pages 1420-1425, July.
    19. Jue Wang & Philip Shapira, 2011. "Funding acknowledgement analysis: an enhanced tool to investigate research sponsorship impacts: the case of nanotechnology," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(3), pages 563-586, June.
    20. Bornmann, Lutz & Daniel, Hans-Dieter, 2010. "Citation speed as a measure to predict the attention an article receives: An investigation of the validity of editorial decisions at Angewandte Chemie International Edition," Journal of Informetrics, Elsevier, vol. 4(1), pages 83-88.
    21. Dorta-González, Pablo & Dorta-González, María Isabel & Santos-Peñate, Dolores Rosa & Suárez-Vega, Rafael, 2014. "Journal topic citation potential and between-field comparisons: The topic normalized impact factor," Journal of Informetrics, Elsevier, vol. 8(2), pages 406-418.
    22. Wolfgang Glänzel & Ronald Rousseau & Lin Zhang, 2012. "A visual representation of relative first-citation times," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(7), pages 1420-1425, July.
    23. Mario Cleves & William W. Gould & Roberto G. Gutierrez & Yulia Marchenko, 2010. "An Introduction to Survival Analysis Using Stata," Stata Press books, StataCorp LP, edition 3, number saus3, March.
    24. Giovanni Abramo & Ciriaco Andrea D’Angelo & Tindaro Cicero, 2012. "What is the appropriate length of the publication period over which to assess research performance?," Scientometrics, Springer;Akadémiai Kiadó, vol. 93(3), pages 1005-1017, December.
    25. Musselin, Christine, 2013. "How peer review empowers the academic profession and university managers: Changes in relationships between the state, universities and the professoriate," Research Policy, Elsevier, vol. 42(5), pages 1165-1173.
    26. Hendrik P. van Dalen & K?ne Henkens, 2005. "Signals in science - On the importance of signaling in gaining attention in science," Scientometrics, Springer;Akadémiai Kiadó, vol. 64(2), pages 209-233, August.
    27. Sarah de Rijcke & Paul F. Wouters & Alex D. Rushforth & Thomas P. Franssen & Björn Hammarfelt, 2016. "Evaluation practices and effects of indicator use—a literature review," Research Evaluation, Oxford University Press, vol. 25(2), pages 161-169.
    28. Jian Wang, 2013. "Citation time window choice for research impact evaluation," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(3), pages 851-872, March.
    29. Bozeman, Barry & Corley, Elizabeth, 2004. "Scientists' collaboration strategies: implications for scientific and technical human capital," Research Policy, Elsevier, vol. 33(4), pages 599-616, May.
    30. Richard Van Noorden, 2010. "Metrics: A profusion of measures," Nature, Nature, vol. 465(7300), pages 864-866, June.
    31. Martín-Martín, Alberto & Orduna-Malea, Enrique & Thelwall, Mike & Delgado López-Cózar, Emilio, 2018. "Google Scholar, Web of Science, and Scopus: A systematic comparison of citations in 252 subject categories," Journal of Informetrics, Elsevier, vol. 12(4), pages 1160-1177.
    32. Elena De la Poza & Paloma Merello & Antonio Barberá & Alberto Celani, 2021. "Universities’ Reporting on SDGs: Using THE Impact Rankings to Model and Measure Their Contribution to Sustainability," Sustainability, MDPI, vol. 13(4), pages 1-28, February.
    33. Agrawal, Ajay & McHale, John & Oettl, Alexander, 2017. "How stars matter: Recruiting and peer effects in evolutionary biology," Research Policy, Elsevier, vol. 46(4), pages 853-867.
    34. Peder Olesen Larsen & Markus Ins, 2010. "The rate of growth in scientific publication and the decline in coverage provided by Science Citation Index," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(3), pages 575-603, September.
    35. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Guoliang Xu & Longchao Xu & Li Jia, 2022. "Research on Mortality Risk of Chinese Older Adults from the Perspective of Social Health," Sustainability, MDPI, vol. 14(24), pages 1-15, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mingyang Wang & Zhenyu Wang & Guangsheng Chen, 2019. "Which can better predict the future success of articles? Bibliometric indices or alternative metrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(3), pages 1575-1595, June.
    2. Wen-Yau Cathy Lin, 2021. "Effects of open access and articles-in-press mechanisms on publishing lag and first-citation speed: a case on energy and fuels journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(6), pages 4841-4869, June.
    3. Bar-Ilan, Judit, 2008. "Informetrics at the beginning of the 21st century—A review," Journal of Informetrics, Elsevier, vol. 2(1), pages 1-52.
    4. Christian Schneijderberg & Nicolai Götze & Lars Müller, 2022. "A study of 25 years of publication outputs in the German academic profession," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(1), pages 1-28, January.
    5. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    6. Guoqiang Liang & Haiyan Hou & Xiaodan Lou & Zhigang Hu, 2019. "Qualifying threshold of “take-off” stage for successfully disseminated creative ideas," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(3), pages 1193-1208, September.
    7. Liang, Guoqiang & Hou, Haiyan & Ding, Ying & Hu, Zhigang, 2020. "Knowledge recency to the birth of Nobel Prize-winning articles: Gender, career stage, and country," Journal of Informetrics, Elsevier, vol. 14(3).
    8. Jeppe Nicolaisen & Tove Faber Frandsen, 2021. "Number of references: a large-scale study of interval ratios," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(1), pages 259-285, January.
    9. Groen-Xu, Moqi & Bös, Gregor & Teixeira, Pedro A. & Voigt, Thomas & Knapp, Bernhard, 2023. "Short-term incentives of research evaluations: Evidence from the UK Research Excellence Framework," Research Policy, Elsevier, vol. 52(6).
    10. Xin Gu & Karen Blackmore, 2017. "Quantitative study on Australian academic science," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 1009-1035, November.
    11. Anna Tietze & Philip Hofmann, 2019. "The h-index and multi-author hm-index for individual researchers in condensed matter physics," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(1), pages 171-185, April.
    12. Wallace, Matthew L. & Ràfols, Ismael, 2018. "Institutional shaping of research priorities: A case study on avian influenza," Research Policy, Elsevier, vol. 47(10), pages 1975-1989.
    13. Cristina López-Duarte & Marta M. Vidal-Suárez & Belén González-Díaz, 2019. "Cross-national distance and international business: an analysis of the most influential recent models," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(1), pages 173-208, October.
    14. Giovanni Abramo & Ciriaco Andrea D’Angelo & Myroslava Hladchenko, 2023. "Assessing the effects of publication requirements for professorship on research performance and publishing behaviour of Ukrainian academics," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(8), pages 4589-4609, August.
    15. Gerson Pech & Catarina Delgado, 2020. "Percentile and stochastic-based approach to the comparison of the number of citations of articles indexed in different bibliographic databases," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(1), pages 223-252, April.
    16. Wang, Jian & Veugelers, Reinhilde & Stephan, Paula, 2017. "Bias against novelty in science: A cautionary tale for users of bibliometric indicators," Research Policy, Elsevier, vol. 46(8), pages 1416-1436.
    17. Andrea Fronzetti Colladon & Ciriaco Andrea D’Angelo & Peter A. Gloor, 2020. "Predicting the future success of scientific publications through social network and semantic analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(1), pages 357-377, July.
    18. Iman Tahamtan & Askar Safipour Afshar & Khadijeh Ahamdzadeh, 2016. "Factors affecting number of citations: a comprehensive review of the literature," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(3), pages 1195-1225, June.
    19. Hou, Jianhua & Yang, Xiucai, 2020. "Social media-based sleeping beauties: Defining, identifying and features," Journal of Informetrics, Elsevier, vol. 14(2).
    20. Rodrigo Costas & Thed N. Leeuwen & Anthony F. J. Raan, 2011. "The “Mendel syndrome” in science: durability of scientific literature and its effects on bibliometric analysis of individual scientists," Scientometrics, Springer;Akadémiai Kiadó, vol. 89(1), pages 177-205, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jsusta:v:13:y:2021:i:17:p:9515-:d:620743. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.