IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v18y2024i3s1751157724000415.html
   My bibliography  Save this article

The fading of status bias during the open peer review process

Author

Listed:
  • Sun, Zhuanlan
  • Pang, Ka Lok
  • Li, Yiwei

Abstract

The growing number of preprints allows reviewers to identify the authors’ identities prior to the peer review process. Yet, it remains unclear whether the preprint exposure of prestigious authors to reviewers is correlated with review features. Here, we employed the linear regression model to examine this relationship. By collecting open peer review reports of 2,059 papers published in Nature Communications in 2019 within the fields of biological and health sciences, we found no obvious difference in review features when the identities of authors with different academic prestige are potentially exposed to reviewers. Specifically, no significant effect was observed on the number of questions raised and the sentiments of the review reports (positivity and subjectivity) in the first round of the peer review process. Moreover, we found no evidence that review features from anonymous reviewers were more positively or subjectively expressed than those with reviewers’ names publicly available. The results persisted even when assuming all papers were under single-blind peer review, which were validated by using the eLife data. This study indicates that papers with both prestigious and less well-known authors are treated equally during the open peer review process, which contributes to the ongoing discourse on the fairness of peer review within the scientific community.

Suggested Citation

  • Sun, Zhuanlan & Pang, Ka Lok & Li, Yiwei, 2024. "The fading of status bias during the open peer review process," Journal of Informetrics, Elsevier, vol. 18(3).
  • Handle: RePEc:eee:infome:v:18:y:2024:i:3:s1751157724000415
    DOI: 10.1016/j.joi.2024.101528
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1751157724000415
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.joi.2024.101528?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Sun, Zhuanlan & Clark Cao, C. & Ma, Chao & Li, Yiwei, 2023. "The academic status of reviewers predicts their language use," Journal of Informetrics, Elsevier, vol. 17(4).
    2. Peter Kardos & Ádám Kun & Csaba Pléh & Ferenc Jordán, 2023. "(How) should researchers publicize their research papers before peer review?," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(3), pages 2019-2023, March.
    3. J. A. Garcia & Rosa Rodriguez-Sánchez & J. Fdez-Valdivia, 2020. "Confirmatory bias in peer review," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(1), pages 517-533, April.
    4. Richard B. Freeman & Wei Huang, 2015. "Collaborating with People Like Me: Ethnic Coauthorship within the United States," Journal of Labor Economics, University of Chicago Press, vol. 33(S1), pages 289-318.
    5. Sun, Zhuanlan, 2024. "Textual features of peer review predict top-cited papers: An interpretable machine learning perspective," Journal of Informetrics, Elsevier, vol. 18(2).
    6. Heckman, James, 2013. "Sample selection bias as a specification error," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 31(3), pages 129-137.
    7. Ma, Chao & Li, Yiwei & Guo, Feng & Si, Kao, 2019. "The citation trap: Papers published at year-end receive systematically fewer citations," Journal of Economic Behavior & Organization, Elsevier, vol. 166(C), pages 667-687.
    8. Ewen Callaway, 2013. "Preprints come to life," Nature, Nature, vol. 503(7475), pages 180-180, November.
    9. Patrick S. Forscher & William T. L. Cox & Markus Brauer & Patricia G. Devine, 2019. "Little race or gender bias in an experiment of initial review of NIH R01 grant proposals," Nature Human Behaviour, Nature, vol. 3(3), pages 257-264, March.
    10. repec:nas:journl:v:115:y:2018:p:4887-4890 is not listed on IDEAS
    11. Vieira, E.S. & Gomes, J.A.N.F., 2010. "Citations to scientific articles: Its distribution and dependence on the article features," Journal of Informetrics, Elsevier, vol. 4(1), pages 1-13.
    12. Wei Huang, 2015. "DO ABCs GET MORE CITATIONS THAN XYZs?," Economic Inquiry, Western Economic Association International, vol. 53(1), pages 773-789, January.
    13. Carole J. Lee & Cassidy R. Sugimoto & Guo Zhang & Blaise Cronin, 2013. "Bias in peer review," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(1), pages 2-17, January.
    14. Sun, Zhuanlan & Liu, Sheng & Li, Yiwei & Ma, Chao, 2023. "Expedited editorial decision in COVID-19 pandemic," Journal of Informetrics, Elsevier, vol. 17(1).
    15. Giangiacomo Bravo & Francisco Grimaldo & Emilia López-Iñesta & Bahar Mehmani & Flaminio Squazzoni, 2019. "The effect of publishing peer review reports on referee behavior in five scholarly journals," Nature Communications, Nature, vol. 10(1), pages 1-8, December.
    16. Carole J. Lee & Cassidy R. Sugimoto & Guo Zhang & Blaise Cronin, 2013. "Bias in peer review," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(1), pages 2-17, January.
    17. Jurgen Huber & Sabiou M. Inoua & Rudolf Kerschbamer & Christian Konig-Kersting & Stefan Palan & Vernon L. Smith, 2022. "Nobel and Novice: Author Prominence Affects Peer Review," Working Papers 22-15, Chapman University, Economic Science Institute.
    18. David Card & Stefano DellaVigna, 2014. "Page Limits on Economics Articles: Evidence from Two Journals," Journal of Economic Perspectives, American Economic Association, vol. 28(3), pages 149-168, Summer.
    19. Zhang, Guangyao & Xu, Shenmeng & Sun, Yao & Jiang, Chunlin & Wang, Xianwen, 2022. "Understanding the peer review endeavor in scientific publishing," Journal of Informetrics, Elsevier, vol. 16(2).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Sun, Zhuanlan & He, Dongjin & Li, Yiwei, 2024. "How the readability of manuscript before journal submission advantages peer review process: Evidence from biomedical scientific publications," Journal of Informetrics, Elsevier, vol. 18(3).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sun, Zhuanlan & Clark Cao, C. & Ma, Chao & Li, Yiwei, 2023. "The academic status of reviewers predicts their language use," Journal of Informetrics, Elsevier, vol. 17(4).
    2. Sun, Zhuanlan & He, Dongjin & Li, Yiwei, 2024. "How the readability of manuscript before journal submission advantages peer review process: Evidence from biomedical scientific publications," Journal of Informetrics, Elsevier, vol. 18(3).
    3. Zhuanlan Sun & C. Clark Cao & Sheng Liu & Yiwei Li & Chao Ma, 2024. "Behavioral consequences of second-person pronouns in written communications between authors and reviewers of scientific papers," Nature Communications, Nature, vol. 15(1), pages 1-12, December.
    4. Ma, Chao & Li, Yiwei & Guo, Feng & Si, Kao, 2019. "The citation trap: Papers published at year-end receive systematically fewer citations," Journal of Economic Behavior & Organization, Elsevier, vol. 166(C), pages 667-687.
    5. Hou, Li & Wu, Qiang & Xie, Yundong, 2024. "Does open identity of peer reviewers positively relate to citations?," Journal of Informetrics, Elsevier, vol. 18(1).
    6. Feng Guo & Chao Ma & Qingling Shi & Qingqing Zong, 2018. "Succinct effect or informative effect: the relationship between title length and the number of citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 1531-1539, September.
    7. Sun, Zhuanlan, 2024. "Textual features of peer review predict top-cited papers: An interpretable machine learning perspective," Journal of Informetrics, Elsevier, vol. 18(2).
    8. Yezhu Wang & Yundong Xie & Dong Wang & Lu Guo & Rongting Zhou, 2022. "Do cover papers get better citations and usage counts? An analysis of 42 journals in cell biology," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(7), pages 3793-3813, July.
    9. Buljan, Ivan & Garcia-Costa, Daniel & Grimaldo, Francisco & Klein, Richard A. & Bakker, Marjan & Marušić, Ana, 2024. "Development and application of a comprehensive glossary for the identification of statistical and methodological concepts in peer review reports," Journal of Informetrics, Elsevier, vol. 18(3).
    10. Cheng, Xi & Wang, Haoran & Tang, Li & Jiang, Weiyan & Zhou, Maotian & Wang, Guoyan, 2024. "Open peer review correlates with altmetrics but not with citations: Evidence from Nature Communications and PLoS One," Journal of Informetrics, Elsevier, vol. 18(3).
    11. Petersen, Alexander M., 2019. "Megajournal mismanagement: Manuscript decision bias and anomalous editor activity at PLOS ONE," Journal of Informetrics, Elsevier, vol. 13(4).
    12. Xie, Yundong & Wu, Qiang & Wang, Yezhu & Hou, Li & Liu, Yuanyuan, 2024. "Does the handling time of scientific papers relate to their academic impact and social attention? Evidence from Nature, Science, and PNAS," Journal of Informetrics, Elsevier, vol. 18(2).
    13. Monica Aniela Zaharie & Marco Seeber, 2018. "Are non-monetary rewards effective in attracting peer reviewers? A natural experiment," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(3), pages 1587-1609, December.
    14. Wenqing Wu & Haixu Xi & Chengzhi Zhang, 2024. "Are the confidence scores of reviewers consistent with the review content? Evidence from top conference proceedings in AI," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(7), pages 4109-4135, July.
    15. Jürgen Janger & Nicole Schmidt-Padickakudy & Anna Strauss-Kollin, 2019. "International Differences in Basic Research Grant Funding. A Systematic Comparison," WIFO Studies, WIFO, number 61664, January.
    16. Kong, Ling & Wang, Dongbo, 2020. "Comparison of citations and attention of cover and non-cover papers," Journal of Informetrics, Elsevier, vol. 14(4).
    17. Rodríguez Sánchez, Isabel & Makkonen, Teemu & Williams, Allan M., 2019. "Peer review assessment of originality in tourism journals: critical perspective of key gatekeepers," Annals of Tourism Research, Elsevier, vol. 77(C), pages 1-11.
    18. Zhentao Liang & Jin Mao & Gang Li, 2023. "Bias against scientific novelty: A prepublication perspective," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 74(1), pages 99-114, January.
    19. Elena Veretennik & Maria Yudkevich, 2023. "Inconsistent quality signals: evidence from the regional journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(6), pages 3675-3701, June.
    20. Meyer, Matthias & Waldkirch, Rüdiger W. & Duscher, Irina & Just, Alexander, 2018. "Drivers of citations: An analysis of publications in “top” accounting journals," CRITICAL PERSPECTIVES ON ACCOUNTING, Elsevier, vol. 51(C), pages 24-46.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:18:y:2024:i:3:s1751157724000415. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joi .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.