IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v128y2023i5d10.1007_s11192-023-04683-9.html
   My bibliography  Save this article

What does open peer review bring to scientific articles? Evidence from PLoS journals

Author

Listed:
  • Chunli Wei

    (Nanjing University)

  • Jingyi Zhao

    (Nanjing University)

  • Jue Ni

    (Nanjing University)

  • Jiang Li

    (Nanjing University)

Abstract

This study examined the impact of open peer review (OPR) on the usage and citations of scientific articles using a dataset of 6441 articles published in six Public Library of Science (PLoS) journals in 2020–2021. We compared OPR articles with their non-OPR counterparts in the same journal to determine whether OPR increased the visibility and citations of the articles. Our results demonstrated a positive association between OPR and higher article page views, saving, sharing, and a greater HTML to PDF conversion rate. However, we also found that OPR articles had a lower PDF to citations conversion rate compared to non-OPR articles. Furthermore, we investigated the effects of OPR on citations across various citation databases, including Web of Science, Scopus, Google Scholar, Semantic Scholar, and Dimensions. Our analysis indicated that OPR had a heterogeneous impact on citations across these databases. These findings provide compelling evidence for stakeholders, such as policymakers, publishers, and researchers, to participate in OPR and promote its adoption in scientific publishing. Additionally, our study underscores the importance of carefully selecting bibliographic databases when assessing the effect of OPR on article citations.

Suggested Citation

  • Chunli Wei & Jingyi Zhao & Jue Ni & Jiang Li, 2023. "What does open peer review bring to scientific articles? Evidence from PLoS journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(5), pages 2763-2776, May.
    • Handle: RePEc:spr:scient:v:128:y:2023:i:5:d:10.1007_s11192-023-04683-9
    DOI: 10.1007/s11192-023-04683-9
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-023-04683-9
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-023-04683-9?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Dietmar Wolfram & Peiling Wang & Adam Hembree & Hyoungjoo Park, 2020. "Open peer review: promoting transparency in open science," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 1033-1051, November.
    2. Bornmann, Lutz & Williams, Richard, 2013. "How to calculate the practical significance of citation impact differences? An empirical example from evaluative institutional bibliometrics using adjusted predictions and marginal effects," Journal of Informetrics, Elsevier, vol. 7(2), pages 562-574.
    3. Jue Ni & Zhenyue Zhao & Yupo Shao & Shuo Liu & Wanlin Li & Yaoze Zhuang & Junmo Qu & Yu Cao & Nayuan Lian & Jiang Li, 2021. "The influence of opening up peer review on the citations of journal articles," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(12), pages 9393-9404, December.
    4. Vincent Larivière & Yves Gingras & Cassidy R. Sugimoto & Andrew Tsou, 2015. "Team size matters: Collaboration and scientific impact since 1900," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(7), pages 1323-1332, July.
    5. Jin-kun Wan & Ping-huan Hua & Ronald Rousseau & Xiu-kun Sun, 2010. "The journal download immediacy index (DII): experiences using a Chinese full-text database," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(3), pages 555-566, March.
    6. Feng Guo & Chao Ma & Qingling Shi & Qingqing Zong, 2018. "Succinct effect or informative effect: the relationship between title length and the number of citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 1531-1539, September.
    7. Alfonso Ibáñez & Concha Bielza & Pedro Larrañaga, 2013. "Relationship among research collaboration, number of documents and number of citations: a case study in Spanish computer science production in 2000–2009," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(2), pages 689-716, May.
    8. Liwen Vaughan & Juan Tang & Rongbin Yang, 2017. "Investigating disciplinary differences in the relationships between citations and downloads," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(3), pages 1533-1545, June.
    9. Ma, Chao & Li, Yiwei & Guo, Feng & Si, Kao, 2019. "The citation trap: Papers published at year-end receive systematically fewer citations," Journal of Economic Behavior & Organization, Elsevier, vol. 166(C), pages 667-687.
    10. Xianwen Wang & Chen Liu & Wenli Mao & Zhichao Fang, 2015. "Erratum to: The open access advantage considering citation, article usage and social media attention," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(3), pages 1149-1149, June.
    11. Susanne Beck & Maral Mahdad & Karin Beukel & Marion Poetz, 2019. "The Value of Scientific Knowledge Dissemination for Scientists—A Value Capture Perspective," Publications, MDPI, vol. 7(3), pages 1-23, July.
    12. Vivek Kumar Singh & Prashasti Singh & Mousumi Karmakar & Jacqueline Leta & Philipp Mayr, 2021. "The journal coverage of Web of Science, Scopus and Dimensions: A comparative analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(6), pages 5113-5142, June.
    13. Lutz Bornmann & Werner Marx & Hermann Schier & Andreas Thor & Hans-Dieter Daniel, 2010. "From black box to white box at open access journals: predictive validity of manuscript reviewing and editorial decisions at Atmospheric Chemistry and Physics," Research Evaluation, Oxford University Press, vol. 19(2), pages 105-118, June.
    14. Giangiacomo Bravo & Francisco Grimaldo & Emilia López-Iñesta & Bahar Mehmani & Flaminio Squazzoni, 2019. "The effect of publishing peer review reports on referee behavior in five scholarly journals," Nature Communications, Nature, vol. 10(1), pages 1-8, December.
    15. Eirini Delikoura & Dimitrios Kouis, 2021. "Open Research Data and Open Peer Review: Perceptions of a Medical and Health Sciences Community in Greece," Publications, MDPI, vol. 9(2), pages 1-19, March.
    16. Iman Tahamtan & Askar Safipour Afshar & Khadijeh Ahamdzadeh, 2016. "Factors affecting number of citations: a comprehensive review of the literature," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(3), pages 1195-1225, June.
    17. Xianwen Wang & Chen Liu & Wenli Mao & Zhichao Fang, 2015. "The open access advantage considering citation, article usage and social media attention," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(2), pages 555-564, May.
    18. Qianjin Zong & Yafen Xie & Jiechun Liang, 2020. "Does open peer review improve citation count? Evidence from a propensity score matching analysis of PeerJ," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 607-623, October.
    19. Xianwen Wang & Wenli Mao & Shenmeng Xu & Chunbo Zhang, 2014. "Usage history of scientific literature: Nature metrics and metrics of Nature publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(3), pages 1923-1933, March.
    20. Bornmann, Lutz & Schier, Hermann & Marx, Werner & Daniel, Hans-Dieter, 2012. "What factors determine citation counts of publications in chemistry besides their quality?," Journal of Informetrics, Elsevier, vol. 6(1), pages 11-18.
    21. Martín-Martín, Alberto & Orduna-Malea, Enrique & Thelwall, Mike & Delgado López-Cózar, Emilio, 2018. "Google Scholar, Web of Science, and Scopus: A systematic comparison of citations in 252 subject categories," Journal of Informetrics, Elsevier, vol. 12(4), pages 1160-1177.
    22. Stefano Mammola & Diego Fontaneto & Alejandro Martínez & Filipe Chichorro, 2021. "Impact of the reference list features on the number of citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(1), pages 785-799, January.
    23. Juan Gorraiz & Christian Gumpenberger & Christian Schlögl, 2014. "Usage versus citation behaviours in four subject areas," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 1077-1095, November.
    24. Ping Ni & Xinying An, 2018. "Relationship between international collaboration papers and their citations from an economic perspective," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(2), pages 863-877, August.
    25. Hendrik P. Van Dalen & Kène Henkens, 2001. "What makes a scientific article influential? The case of demographers," Scientometrics, Springer;Akadémiai Kiadó, vol. 50(3), pages 455-482, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Martorell Cunil, Onofre & Otero González, Luis & Durán Santomil, Pablo & Mulet Forteza, Carlos, 2023. "How to accomplish a highly cited paper in the tourism, leisure and hospitality field," Journal of Business Research, Elsevier, vol. 157(C).
    2. Hou, Li & Wu, Qiang & Xie, Yundong, 2024. "Does open identity of peer reviewers positively relate to citations?," Journal of Informetrics, Elsevier, vol. 18(1).
    3. Wencan Tian & Yongzhen Wang & Zhigang Hu & Ruonan Cai & Guangyao Zhang & Xianwen Wang, 2024. "Does Granger causality exist between article usage and publication counts? A topic-level time-series evidence from IEEE Xplore," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(6), pages 3285-3302, June.
    4. Bikun Chen, 2018. "Usage pattern comparison of the same scholarly articles between Web of Science (WoS) and Springer," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 519-537, April.
    5. Rayanne Barros Setubal & Daniel Silva Farias & Clarice Casa Nova & Anna Carolina Fornero Aguiar & Tauany Aparecida Silva Santa Rosa Rodrigues & Rafael Teixeira Santos Lira & Anderson Luiz Vargas Ferre, 2022. "Microwave effect: analyzing citations from classic theories and their reinventions—a case study from a classic paper in aquatic ecology—Brooks & Dodson, 1965," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(8), pages 4751-4767, August.
    6. Barbara McGillivray & Mathias Astell, 2019. "The relationship between usage and citations in an open access mega-journal," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(2), pages 817-838, November.
    7. Kong, Ling & Wang, Dongbo, 2020. "Comparison of citations and attention of cover and non-cover papers," Journal of Informetrics, Elsevier, vol. 14(4).
    8. Zhentao Liang & Jin Mao & Kun Lu & Gang Li, 2021. "Finding citations for PubMed: a large-scale comparison between five freely available bibliographic data sources," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(12), pages 9519-9542, December.
    9. Mingyang Wang & Zhenyu Wang & Guangsheng Chen, 2019. "Which can better predict the future success of articles? Bibliometric indices or alternative metrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(3), pages 1575-1595, June.
    10. Liu, Jialin & Chen, Hongkan & Liu, Zhibo & Bu, Yi & Gu, Weiye, 2022. "Non-linearity between referencing behavior and citation impact: A large-scale, discipline-level analysis," Journal of Informetrics, Elsevier, vol. 16(3).
    11. Guoqiang Liang & Haiyan Hou & Qiao Chen & Zhigang Hu, 2020. "Diffusion and adoption: an explanatory model of “question mark” and “rising star” articles," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(1), pages 219-232, July.
    12. Fan, Lingxu & Guo, Lei & Wang, Xinhua & Xu, Liancheng & Liu, Fangai, 2022. "Does the author’s collaboration mode lead to papers’ different citation impacts? An empirical analysis based on propensity score matching," Journal of Informetrics, Elsevier, vol. 16(4).
    13. Zhiqi Wang & Wolfgang Glänzel & Yue Chen, 2020. "The impact of preprints in Library and Information Science: an analysis of citations, usage and social attention indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 1403-1423, November.
    14. Raminta Pranckutė, 2021. "Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World," Publications, MDPI, vol. 9(1), pages 1-59, March.
    15. Brady D. Lund & Sanjay Kumar Maurya, 2020. "The relationship between highly-cited papers and the frequency of citations to other papers within-issue among three top information science journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2491-2504, December.
    16. Giovanni Abramo & Ciriaco Andrea D’Angelo & Leonardo Grilli, 2024. "The role of non-scientific factors vis-à-vis the quality of publications in determining their scholarly impact," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(8), pages 5003-5019, August.
    17. Hongquan Shen & Juan Xie & Jiang Li & Ying Cheng, 2021. "The correlation between scientific collaboration and citation count at the paper level: a meta-analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(4), pages 3443-3470, April.
    18. Ha, Taehyun, 2022. "An explainable artificial-intelligence-based approach to investigating factors that influence the citation of papers," Technological Forecasting and Social Change, Elsevier, vol. 184(C).
    19. Jue Ni & Zhenyue Zhao & Yupo Shao & Shuo Liu & Wanlin Li & Yaoze Zhuang & Junmo Qu & Yu Cao & Nayuan Lian & Jiang Li, 2021. "The influence of opening up peer review on the citations of journal articles," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(12), pages 9393-9404, December.
    20. Muhammad Salman Khan & Muhammad Younas, 2017. "Analyzing readers behavior in downloading articles from IEEE digital library: a study of two selected journals in the field of education," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(3), pages 1523-1537, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:128:y:2023:i:5:d:10.1007_s11192-023-04683-9. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.