IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v129y2024i11d10.1007_s11192-024-05167-0.html
   My bibliography  Save this article

Do Changes in Journal Rank Influence Publication Output? Evidence from China

Author

Listed:
  • Zhuanlan Sun

    (Nanjing University of Posts and Telecommunications)

  • Chenwei Zhang

    (The University of Hong Kong)

  • Ka Lok Pang

    (Lingnan University)

  • Ying Tang

    (Nanjing University of Posts and Telecommunications)

  • Yiwei Li

    (Lingnan University)

Abstract

Despite considerable progress in understanding the journal evaluation system in China, empirical evidence remains limited regarding the impact of changes in journal rank (CJR) on scientific output. By employing the difference-in-differences (DID) framework, we exploit panel data from 2015 to 2019 to examine the effect of changes in journal ranks on the number of publications by Chinese researchers. Our analysis involves comparing two groups—journals that experienced a change in ranking and journals that did not—before and after the change in ranking. Our analysis reveals a statistically significant negative effect. The results suggest that CJR has led to a 14.81% decrease in the number of publications per 100, relative to the sample mean value. The observed negative impact is consistently confirmed through robustness tests that involve excluding journals that do not adhere to best practices, removing retracted publications from the calculation of publication numbers, and randomly selecting journals with changed ranks for estimation. We also observed that CJR exhibits a widespread but unequal effect. The negative effect is particularly pronounced in the academic domains of life sciences and physical sciences, in journals that experience declines in rank, and in less-prestigious universities. We contribute to the literature on how changes in journal rankings affect researchers’ academic behavior and the determinants of scholarly publication productivity.

Suggested Citation

  • Zhuanlan Sun & Chenwei Zhang & Ka Lok Pang & Ying Tang & Yiwei Li, 2024. "Do Changes in Journal Rank Influence Publication Output? Evidence from China," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(11), pages 7035-7054, November.
    • Handle: RePEc:spr:scient:v:129:y:2024:i:11:d:10.1007_s11192-024-05167-0
    DOI: 10.1007/s11192-024-05167-0
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-024-05167-0
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-024-05167-0?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Liwei Zhang & Liang Ma, 2023. "Is open science a double-edged sword?: data sharing and the changing citation pattern of Chinese economics articles," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(5), pages 2803-2818, May.
    2. Piotr Śpiewanowski & Oleksandr Talavera, 2021. "Journal rankings and publication strategy," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(4), pages 3227-3242, April.
    3. Jacob, Brian A. & Lefgren, Lars, 2011. "The impact of research grant funding on scientific productivity," Journal of Public Economics, Elsevier, vol. 95(9), pages 1168-1177.
    4. Eliana La Ferrara & Alberto Chong & Suzanne Duryea, 2012. "Soap Operas and Fertility: Evidence from Brazil," American Economic Journal: Applied Economics, American Economic Association, vol. 4(4), pages 1-31, October.
    5. Jacob, Brian A. & Lefgren, Lars, 2011. "The impact of NIH postdoctoral training grants on scientific productivity," Research Policy, Elsevier, vol. 40(6), pages 864-874, July.
    6. Fry, Caroline V. & Lynham, John & Tran, Shannon, 2023. "Ranking researchers: Evidence from Indonesia," Research Policy, Elsevier, vol. 52(5).
    7. Claudiu Vasile Kifor & Ana Maria Benedek & Ioan Sîrbu & Roxana Florența Săvescu, 2023. "Institutional drivers of research productivity: a canonical multivariate analysis of Romanian public universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(4), pages 2233-2258, April.
    8. Cheng Peng & Zhepeng (Lionel) Li & Chaojiang Wu, 2023. "Researcher geographic mobility and publication productivity: an investigation into individual and institutional characteristics and the roles of academicians," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(1), pages 379-406, January.
    9. Hicks, Diana, 2012. "Performance-based university research funding systems," Research Policy, Elsevier, vol. 41(2), pages 251-261.
    10. Roula Inglesi-Lotz & Anastassios Pouris, 2011. "Scientometric impact assessment of a research policy instrument: the case of rating researchers on scientific outputs in South Africa," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(3), pages 747-760, September.
    11. Yu, Nannan & Dong, Yueyan & de Jong, Martin, 2022. "A helping hand from the government? How public research funding affects academic output in less-prestigious universities in China," Research Policy, Elsevier, vol. 51(10).
    12. Gonzalez-Brambila, Claudia & Veloso, Francisco M., 2007. "The determinants of research output and impact: A study of Mexican researchers," Research Policy, Elsevier, vol. 36(7), pages 1035-1051, September.
    13. Goodman-Bacon, Andrew, 2021. "Difference-in-differences with variation in treatment timing," Journal of Econometrics, Elsevier, vol. 225(2), pages 254-277.
    14. Alexander, John C & Mabry, Rodney H, 1994. "Relative Significance of Journals, Authors, and Articles Cited in Financial Research," Journal of Finance, American Finance Association, vol. 49(2), pages 697-712, June.
    15. João M. Santos & Hugo Horta & Huan Li, 2022. "Correction to: Are the strategic research agendas of researchers in the social sciences determinants of research productivity?," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(7), pages 3749-3749, July.
    16. Weihua Li & Tomaso Aste & Fabio Caccioli & Giacomo Livan, 2019. "Early coauthorship with top scientists predicts success in academic careers," Nature Communications, Nature, vol. 10(1), pages 1-9, December.
    17. Martin Szomszor & David A. Pendlebury & Jonathan Adams, 2020. "How much is too much? The difference between research influence and self-citation excess," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(2), pages 1119-1147, May.
    18. João M. Santos & Hugo Horta & Huan Li, 2022. "Are the strategic research agendas of researchers in the social sciences determinants of research productivity?," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(7), pages 3719-3747, July.
    19. Drivas, Kyriakos & Kremmydas, Dimitris, 2020. "The Matthew effect of a journal's ranking," Research Policy, Elsevier, vol. 49(4).
    20. Wei Chen & Qin-Rui Xing & Hui Wang & Tao Wang, 2018. "Retracted publications in the biomedical literature with authors from mainland China," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(1), pages 217-227, January.
    21. Zhuanlan Sun & C. Clark Cao & Sheng Liu & Yiwei Li & Chao Ma, 2024. "Behavioral consequences of second-person pronouns in written communications between authors and reviewers of scientific papers," Nature Communications, Nature, vol. 15(1), pages 1-12, December.
    22. Demir, Selcuk Besir, 2018. "Predatory journals: Who publishes in them and why?," Journal of Informetrics, Elsevier, vol. 12(4), pages 1296-1311.
    23. Sun, Zhuanlan & Liu, Sheng & Li, Yiwei & Ma, Chao, 2023. "Expedited editorial decision in COVID-19 pandemic," Journal of Informetrics, Elsevier, vol. 17(1).
    24. Marcel Ausloos & Olgica Nedič & Aleksandar Dekanski, 2019. "Correlations between submission and acceptance of papers in peer review journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(1), pages 279-302, April.
    25. Ana C. M. Brito & Filipi N. Silva & Diego R. Amancio, 2023. "Analyzing the influence of prolific collaborations on authors productivity and visibility," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(4), pages 2471-2487, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Li, Meiling & Wang, Yang & Du, Haifeng & Bai, Aruhan, 2024. "Motivating innovation: The impact of prestigious talent funding on junior scientists," Research Policy, Elsevier, vol. 53(9).
    2. Yuret, Tolga, 2017. "Do researchers pay attention to publication subsidies?," Journal of Informetrics, Elsevier, vol. 11(2), pages 423-434.
    3. Lu Liu & Benjamin F. Jones & Brian Uzzi & Dashun Wang, 2023. "Data, measurement and empirical methods in the science of science," Nature Human Behaviour, Nature, vol. 7(7), pages 1046-1058, July.
    4. Young-Hwan Lee, 2021. "Determinants of research productivity in Korean Universities: the role of research funding," The Journal of Technology Transfer, Springer, vol. 46(5), pages 1462-1486, October.
    5. Benavente, José Miguel & Crespi, Gustavo & Figal Garone, Lucas & Maffioli, Alessandro, 2012. "The impact of national research funds: A regression discontinuity approach to the Chilean FONDECYT," Research Policy, Elsevier, vol. 41(8), pages 1461-1475.
    6. Wu, Jiang & Ou, Guiyan & Liu, Xiaohui & Dong, Ke, 2022. "How does academic education background affect top researchers’ performance? Evidence from the field of artificial intelligence," Journal of Informetrics, Elsevier, vol. 16(2).
    7. Chaojiang Wu & Erjia Yan & Yongjun Zhu & Kai Li, 2021. "Gender imbalance in the productivity of funded projects: A study of the outputs of National Institutes of Health R01 grants," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 72(11), pages 1386-1399, November.
    8. Chandler, Vincent, 2018. "Short and long-term impacts of an increase in graduate funding," Economics of Education Review, Elsevier, vol. 62(C), pages 104-112.
    9. Mohamed Boufarss & Mikael Laakso, 2020. "Open Sesame? Open access priorities, incentives, and policies among higher education institutions in the United Arab Emirates," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(2), pages 1553-1577, August.
    10. Karacic, Domagoj & Miskulin, Ivan & Serdarusic, Hrvoje, 2016. "State Investment In Science And Scientific Productivity Of Universities," UTMS Journal of Economics, University of Tourism and Management, Skopje, Macedonia, vol. 7(1), pages 37-48.
    11. Abramo, Giovanni & D'Angelo, Ciriaco Andrea & Grilli, Leonardo, 2021. "The effects of citation-based research evaluation schemes on self-citation behavior," Journal of Informetrics, Elsevier, vol. 15(4).
    12. Jun Zhang & Xiaoyan Su & Yifei Wang, 2024. "A Qualitative Study on the Relationship between Faculty Mobility and Scientific Impact: Toward the Sustainable Development of Higher Education," Sustainability, MDPI, vol. 16(17), pages 1-18, September.
    13. Lawson, Cornelia & Salter, Ammon, 2023. "Exploring the effect of overlapping institutional applications on panel decision-making," Research Policy, Elsevier, vol. 52(9).
    14. Park, Hyunwoo & Lee, Jeongsik (Jay) & Kim, Byung-Cheol, 2015. "Project selection in NIH: A natural experiment from ARRA," Research Policy, Elsevier, vol. 44(6), pages 1145-1159.
    15. Dong, Siyuan & Wang, Na & Fan, Chengwei & Chen, Shi & Zhang, Lei, 2024. "E-commerce and rural women entrepreneurship ——Based on the quasi-natural experiment of “comprehensive demonstration policy” for E-commerce in rural areas," Economic Analysis and Policy, Elsevier, vol. 83(C), pages 749-765.
    16. Xufei Zhang & Hongsheng Fang & Lin Guo, 2023. "Corporate Endowment Insurance Fee Reduction and Employee Wages: Evidence from China," China & World Economy, Institute of World Economics and Politics, Chinese Academy of Social Sciences, vol. 31(2), pages 192-222, March.
    17. Dzieżyc, Maciej & Kazienko, Przemysław, 2022. "Effectiveness of research grants funded by European Research Council and Polish National Science Centre," Journal of Informetrics, Elsevier, vol. 16(1).
    18. Kim, Woojin, 2022. "Television and American consumerism," Journal of Public Economics, Elsevier, vol. 208(C).
    19. Marielle Non & Jeroen van Honk & Vince van Houten & Inge van der Weijden & Thed van Leeuwen, 2022. "Getting off to a flying start? The effects of an early-career international mobility grant on scientific performance," CPB Discussion Paper 443, CPB Netherlands Bureau for Economic Policy Analysis.
    20. Yun Yang & Feng Hao & Xingchen Meng, 2024. "How does the national e-commerce demonstration city pilot policy boost economic growth? Evidence from China," Empirica, Springer;Austrian Institute for Economic Research;Austrian Economic Association, vol. 51(4), pages 929-975, November.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:129:y:2024:i:11:d:10.1007_s11192-024-05167-0. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.