IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v18y2024i1s1751157724000026.html
   My bibliography  Save this article

Does open identity of peer reviewers positively relate to citations?

Author

Listed:
  • Hou, Li
  • Wu, Qiang
  • Xie, Yundong

Abstract

Referees play an essential role in the peer review process, significantly contributing to improving the quality of scholarly publications. In the context that the Open Science movement is gaining increasing attention and support, the open identity of peer reviewers has become a crucial aspect requiring more relevant research. To enhance the comprehension of the relatively unexplored phenomenon of publishing reviewer identities, this study investigates how such identification and the academic performance of reviewers identified in the acknowledgements of articles relate to these articles’ citations. Our sample comprises 1,120 articles within the field of Biological Sciences and published in Nature between 2016 and 2020. The publication and citation history for 787 identified reviewers were obtained from Scopus to measure their academic performance. Based on Negative Binomial Regression results, we found that the presence of at least one reviewer's identity is not statistically associated with citations, whereas articles with all reviewers’ identities published tend to receive fewer citations. However, our dataset supports a significant positive correlation between the academic performance of recognised reviewers and the citation impact of the articles they reviewed. These findings have significant implications for the scholarly community and publication policies.

Suggested Citation

  • Hou, Li & Wu, Qiang & Xie, Yundong, 2024. "Does open identity of peer reviewers positively relate to citations?," Journal of Informetrics, Elsevier, vol. 18(1).
    • Handle: RePEc:eee:infome:v:18:y:2024:i:1:s1751157724000026
    DOI: 10.1016/j.joi.2024.101489
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1751157724000026
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.joi.2024.101489?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Zhenquan Lin & Shanci Hou & Jinshan Wu, 2016. "The correlation between editorial delay and the ratio of highly cited papers in Nature, Science and Physical Review Letters," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(3), pages 1457-1464, June.
    2. Dietmar Wolfram & Peiling Wang & Adam Hembree & Hyoungjoo Park, 2020. "Open peer review: promoting transparency in open science," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 1033-1051, November.
    3. Mingers, John & Leydesdorff, Loet, 2015. "A review of theory and practice in scientometrics," European Journal of Operational Research, Elsevier, vol. 246(1), pages 1-19.
    4. Syed Hasan & Robert Breunig, 2021. "Article length and citation outcomes," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 7583-7608, September.
    5. Bianchi, Federico & García-Costa, Daniel & Grimaldo, Francisco & Squazzoni, Flaminio, 2022. "Measuring the effect of reviewers on manuscript change: A study on a sample of submissions to Royal Society journals (2006–2017)," Journal of Informetrics, Elsevier, vol. 16(3).
    6. Jue Ni & Zhenyue Zhao & Yupo Shao & Shuo Liu & Wanlin Li & Yaoze Zhuang & Junmo Qu & Yu Cao & Nayuan Lian & Jiang Li, 2021. "The influence of opening up peer review on the citations of journal articles," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(12), pages 9393-9404, December.
    7. Juan A Crespo & Yungrong Li & Javier Ruiz–Castillo, 2013. "The Measurement of the Effect on Citation Inequality of Differences in Citation Practices across Scientific Fields," PLOS ONE, Public Library of Science, vol. 8(3), pages 1-9, March.
    8. Weihua Li & Tomaso Aste & Fabio Caccioli & Giacomo Livan, 2019. "Early coauthorship with top scientists predicts success in academic careers," Nature Communications, Nature, vol. 10(1), pages 1-9, December.
    9. Feng Guo & Chao Ma & Qingling Shi & Qingqing Zong, 2018. "Succinct effect or informative effect: the relationship between title length and the number of citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 1531-1539, September.
    10. Si Shen & Ronald Rousseau & Dongbo Wang & Danhao Zhu & Huoyu Liu & Ruilun Liu, 2015. "Editorial delay and its relation to subsequent citations: the journals Nature, Science and Cell," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(3), pages 1867-1873, December.
    11. Ma, Chao & Li, Yiwei & Guo, Feng & Si, Kao, 2019. "The citation trap: Papers published at year-end receive systematically fewer citations," Journal of Economic Behavior & Organization, Elsevier, vol. 166(C), pages 667-687.
    12. Dietmar Wolfram & Peiling Wang & Fuad Abuzahra, 2021. "An exploration of referees’ comments published in open peer review journals: The characteristics of review language and the association between review scrutiny and citations [Peer Review for Journa," Research Evaluation, Oxford University Press, vol. 30(3), pages 314-322.
    13. Samuel F. Way & Allison C. Morgan & Daniel B. Larremore & Aaron Clauset, 2019. "Productivity, prominence, and the effects of academic environment," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 116(22), pages 10729-10733, May.
    14. Danielle H. Lee, 2019. "Predicting the research performance of early career scientists," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(3), pages 1481-1504, December.
    15. Yundong Xie & Qiang Wu & Xingchen Li, 2019. "Editorial team scholarly index (ETSI): an alternative indicator for evaluating academic journal reputation," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(3), pages 1333-1349, September.
    16. Vieira, E.S. & Gomes, J.A.N.F., 2010. "Citations to scientific articles: Its distribution and dependence on the article features," Journal of Informetrics, Elsevier, vol. 4(1), pages 1-13.
    17. Reingewertz, Yaniv & Lutmar, Carmela, 2018. "Academic in-group bias: An empirical examination of the link between author and journal affiliation," Journal of Informetrics, Elsevier, vol. 12(1), pages 74-86.
    18. Dondio, Pierpaolo & Casnici, Niccolò & Grimaldo, Francisco & Gilbert, Nigel & Squazzoni, Flaminio, 2019. "The “invisible hand” of peer review: The implications of author-referee networks on peer review in a scholarly journal," Journal of Informetrics, Elsevier, vol. 13(2), pages 708-716.
    19. Donner, Paul, 2018. "Effect of publication month on citation impact," Journal of Informetrics, Elsevier, vol. 12(1), pages 330-343.
    20. Lorna Wildgaard & Jesper W. Schneider & Birger Larsen, 2014. "A review of the characteristics of 108 author-level bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 125-158, October.
    21. Gaulé, Patrick & Maystre, Nicolas, 2011. "Getting cited: Does open access help?," Research Policy, Elsevier, vol. 40(10), pages 1332-1338.
      • Patrick Gaulé & Nicolas Maystre, 2008. "Getting cited: does open access help?," CEMI Working Papers cemi-workingpaper-2008-00, Ecole Polytechnique Fédérale de Lausanne, Collège du Management de la Technologie, Management of Technology and Entrepreneurship Institute, Chaire en Economie et Management de l'Innovation.
    22. Giangiacomo Bravo & Francisco Grimaldo & Emilia López-Iñesta & Bahar Mehmani & Flaminio Squazzoni, 2019. "The effect of publishing peer review reports on referee behavior in five scholarly journals," Nature Communications, Nature, vol. 10(1), pages 1-8, December.
    23. Dag W. Aksnes & Liv Langfeldt & Paul Wouters, 2019. "Citations, Citation Indicators, and Research Quality: An Overview of Basic Concepts and Theories," SAGE Open, , vol. 9(1), pages 21582440198, February.
    24. Qianjin Zong & Yafen Xie & Jiechun Liang, 2020. "Does open peer review improve citation count? Evidence from a propensity score matching analysis of PeerJ," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 607-623, October.
    25. Mingkun Wei, 2020. "Research on impact evaluation of open access journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(2), pages 1027-1049, February.
    26. Vicente-Saez, Ruben & Martinez-Fuentes, Clara, 2018. "Open Science now: A systematic literature review for an integrated definition," Journal of Business Research, Elsevier, vol. 88(C), pages 428-436.
    27. Yassine Gargouri & Chawki Hajjem & Vincent Larivière & Yves Gingras & Les Carr & Tim Brody & Stevan Harnad, 2010. "Self-Selected or Mandated, Open Access Increases Citation Impact for Higher Quality Research," PLOS ONE, Public Library of Science, vol. 5(10), pages 1-12, October.
    28. Zhang, Guangyao & Xu, Shenmeng & Sun, Yao & Jiang, Chunlin & Wang, Xianwen, 2022. "Understanding the peer review endeavor in scientific publishing," Journal of Informetrics, Elsevier, vol. 16(2).
    29. Yajie Zhang & Qiang Yu, 2020. "What is the best article publishing strategy for early career scientists?," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(1), pages 397-408, January.
    30. Ronald N. Kostoff, 2007. "The difference between highly and poorly cited medical articles in the journal Lancet," Scientometrics, Springer;Akadémiai Kiadó, vol. 72(3), pages 513-520, September.
    31. Fan, Lingxu & Guo, Lei & Wang, Xinhua & Xu, Liancheng & Liu, Fangai, 2022. "Does the author’s collaboration mode lead to papers’ different citation impacts? An empirical analysis based on propensity score matching," Journal of Informetrics, Elsevier, vol. 16(4).
    32. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Li Hou & Qiang Wu & Yundong Xie, 2022. "Does early publishing in top journals really predict long-term scientific success in the business field?," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(11), pages 6083-6107, November.
    2. Xie, Yundong & Wu, Qiang & Wang, Yezhu & Hou, Li & Liu, Yuanyuan, 2024. "Does the handling time of scientific papers relate to their academic impact and social attention? Evidence from Nature, Science, and PNAS," Journal of Informetrics, Elsevier, vol. 18(2).
    3. Chunli Wei & Jingyi Zhao & Jue Ni & Jiang Li, 2023. "What does open peer review bring to scientific articles? Evidence from PLoS journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(5), pages 2763-2776, May.
    4. Yezhu Wang & Yundong Xie & Dong Wang & Lu Guo & Rongting Zhou, 2022. "Do cover papers get better citations and usage counts? An analysis of 42 journals in cell biology," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(7), pages 3793-3813, July.
    5. Sun, Zhuanlan, 2024. "Textual features of peer review predict top-cited papers: An interpretable machine learning perspective," Journal of Informetrics, Elsevier, vol. 18(2).
    6. Brady D. Lund & Sanjay Kumar Maurya, 2020. "The relationship between highly-cited papers and the frequency of citations to other papers within-issue among three top information science journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2491-2504, December.
    7. Kong, Ling & Wang, Dongbo, 2020. "Comparison of citations and attention of cover and non-cover papers," Journal of Informetrics, Elsevier, vol. 14(4).
    8. Sergio Copiello, 2019. "The open access citation premium may depend on the openness and inclusiveness of the indexing database, but the relationship is controversial because it is ambiguous where the open access boundary lie," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(2), pages 995-1018, November.
    9. Bornmann, Lutz & Haunschild, Robin & Mutz, Rüdiger, 2020. "Should citations be field-normalized in evaluative bibliometrics? An empirical analysis based on propensity score matching," Journal of Informetrics, Elsevier, vol. 14(4).
    10. Raminta Pranckutė, 2021. "Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World," Publications, MDPI, vol. 9(1), pages 1-59, March.
    11. Zhentao Liang & Jin Mao & Gang Li, 2023. "Bias against scientific novelty: A prepublication perspective," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 74(1), pages 99-114, January.
    12. Borenstein, Denis & Perlin, Marcelo S. & Imasato, Takeyoshi, 2022. "The Academic Inbreeding Controversy: Analysis and Evidence from Brazil," Journal of Informetrics, Elsevier, vol. 16(2).
    13. Mingkun Wei & Abdolreza Noroozi Chakoli, 2020. "Evaluating the relationship between the academic and social impact of open access books based on citation behaviors and social media attention," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2401-2420, December.
    14. Martorell Cunil, Onofre & Otero González, Luis & Durán Santomil, Pablo & Mulet Forteza, Carlos, 2023. "How to accomplish a highly cited paper in the tourism, leisure and hospitality field," Journal of Business Research, Elsevier, vol. 157(C).
    15. Liu, Jialin & Chen, Hongkan & Liu, Zhibo & Bu, Yi & Gu, Weiye, 2022. "Non-linearity between referencing behavior and citation impact: A large-scale, discipline-level analysis," Journal of Informetrics, Elsevier, vol. 16(3).
    16. Andrea Bonaccorsi & Tindaro Cicero & Peter Haddawy & Saeed-UL Hassan, 2017. "Explaining the transatlantic gap in research excellence," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(1), pages 217-241, January.
    17. Sun, Zhuanlan & Clark Cao, C. & Ma, Chao & Li, Yiwei, 2023. "The academic status of reviewers predicts their language use," Journal of Informetrics, Elsevier, vol. 17(4).
    18. Elizabeth S. Vieira, 2023. "The influence of research collaboration on citation impact: the countries in the European Innovation Scoreboard," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(6), pages 3555-3579, June.
    19. Katchanov, Yurij L. & Markova, Yulia V. & Shmatko, Natalia A., 2019. "The distinction machine: Physics journals from the perspective of the Kolmogorov–Smirnov statistic," Journal of Informetrics, Elsevier, vol. 13(4).
    20. Sepideh Fahimifar & Khadijeh Mousavi & Fatemeh Mozaffari & Marcel Ausloos, 2023. "Identification of the most important external features of highly cited scholarly papers through 3 (i.e., Ridge, Lasso, and Boruta) feature selection data mining methods," Quality & Quantity: International Journal of Methodology, Springer, vol. 57(4), pages 3685-3712, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:18:y:2024:i:1:s1751157724000026. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joi .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.