IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v125y2020i2d10.1007_s11192-020-03488-4.html
   My bibliography  Save this article

Open peer review: promoting transparency in open science

Author

Listed:
  • Dietmar Wolfram

    (University of Wisconsin-Milwaukee)

  • Peiling Wang

    (University of Tennessee)

  • Adam Hembree

    (University of Tennessee)

  • Hyoungjoo Park

    (University of Wisconsin-Milwaukee)

Abstract

Open peer review (OPR), where review reports and reviewers’ identities are published alongside the articles, represents one of the last aspects of the open science movement to be widely embraced, although its adoption has been growing since the turn of the century. This study provides the first comprehensive investigation of OPR adoption, its early adopters and the implementation approaches used. Current bibliographic databases do not systematically index OPR journals, nor do the OPR journals clearly state their policies on open identities and open reports. Using various methods, we identified 617 OPR journals that published at least one article with open identities or open reports as of 2019 and analyzed their wide-ranging implementations to derive emerging OPR practices. The findings suggest that: (1) there has been a steady growth in OPR adoption since 2001, when 38 journals initially adopted OPR, with more rapid growth since 2017; (2) OPR adoption is most prevalent in medical and scientific disciplines (79.9%); (3) five publishers are responsible for 81% of the identified OPR journals; (4) early adopter publishers have implemented OPR in different ways, resulting in different levels of transparency. Across the variations in OPR implementations, two important factors define the degree of transparency: open identities and open reports. Open identities may include reviewer names and affiliation as well as credentials; open reports may include timestamped review histories consisting of referee reports and author rebuttals or a letter from the editor integrating reviewers’ comments. When and where open reports can be accessed are also important factors indicating the OPR transparency level. Publishers of optional OPR journals should add metric data in their annual status reports.

Suggested Citation

  • Dietmar Wolfram & Peiling Wang & Adam Hembree & Hyoungjoo Park, 2020. "Open peer review: promoting transparency in open science," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 1033-1051, November.
  • Handle: RePEc:spr:scient:v:125:y:2020:i:2:d:10.1007_s11192-020-03488-4
    DOI: 10.1007/s11192-020-03488-4
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-020-03488-4
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s11192-020-03488-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Lutz Bornmann & Markus Wolf & Hans-Dieter Daniel, 2012. "Closed versus open reviewing of journal manuscripts: how far do comments differ in language use?," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(3), pages 843-856, June.
    2. Adrian Mulligan & Louise Hall & Ellen Raphael, 2013. "Peer review in a changing world: An international study measuring the attitudes of researchers," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(1), pages 132-161, January.
    3. Adrian Mulligan & Louise Hall & Ellen Raphael, 2013. "Peer review in a changing world: An international study measuring the attitudes of researchers," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(1), pages 132-161, January.
    4. Editorial Article, 0. "The Information for Authors," Economics of Contemporary Russia, Regional Public Organization for Assistance to the Development of Institutions of the Department of Economics of the Russian Academy of Sciences, issue 2.
    5. Julia Gross & John Charles Ryan, 2015. "Landscapes of Research: Perceptions of Open Access (OA) Publishing in the Arts and Humanities," Publications, MDPI, vol. 3(2), pages 1-24, April.
    6. Editorial Article, 0. "The Information for Authors," Economics of Contemporary Russia, Regional Public Organization for Assistance to the Development of Institutions of the Department of Economics of the Russian Academy of Sciences, issue 4.
    7. Carole J. Lee & Cassidy R. Sugimoto & Guo Zhang & Blaise Cronin, 2013. "Bias in peer review," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(1), pages 2-17, January.
    8. Editorial Article, 0. "The Information for Authors," Economics of Contemporary Russia, Regional Public Organization for Assistance to the Development of Institutions of the Department of Economics of the Russian Academy of Sciences, issue 3.
    9. Editorial Article, 0. "The Information for Authors," Economics of Contemporary Russia, Regional Public Organization for Assistance to the Development of Institutions of the Department of Economics of the Russian Academy of Sciences, issue 1.
    10. Editorial Article, 0. "The Information for Authors," Economics of Contemporary Russia, Regional Public Organization for Assistance to the Development of Institutions of the Department of Economics of the Russian Academy of Sciences, issue 4.
    11. Carole J. Lee & Cassidy R. Sugimoto & Guo Zhang & Blaise Cronin, 2013. "Bias in peer review," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(1), pages 2-17, January.
    12. Editorial Article, 0. "The Information for Authors," Economics of Contemporary Russia, Regional Public Organization for Assistance to the Development of Institutions of the Department of Economics of the Russian Academy of Sciences, issue 3.
    13. Editorial Article, 0. "The Information for Authors," Economics of Contemporary Russia, Regional Public Organization for Assistance to the Development of Institutions of the Department of Economics of the Russian Academy of Sciences, issue 2.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Hüttel, Silke & Hess, Sebastian, 2023. "Lessons from the p-value debate and the replication crisis for "open Q science" – the editor's perspective or: will the revolution devour its children?," DARE Discussion Papers 2302, Georg-August University of Göttingen, Department of Agricultural Economics and Rural Development (DARE).
    2. Eirini Delikoura & Dimitrios Kouis, 2021. "Open Research Data and Open Peer Review: Perceptions of a Medical and Health Sciences Community in Greece," Publications, MDPI, vol. 9(2), pages 1-19, March.
    3. Sh Moradi & S Abdi, 2023. "Open science–related policies in Europe," Science and Public Policy, Oxford University Press, vol. 50(3), pages 521-530.
    4. Chunli Wei & Jingyi Zhao & Jue Ni & Jiang Li, 2023. "What does open peer review bring to scientific articles? Evidence from PLoS journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(5), pages 2763-2776, May.
    5. Simon Mahony, 2022. "Toward openness and transparency to better facilitate knowledge creation," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 73(10), pages 1474-1488, October.
    6. Ahmad Yaman Abdin & Muhammad Jawad Nasim & Yannick Ney & Claus Jacob, 2021. "The Pioneering Role of Sci in Post Publication Public Peer Review (P4R)," Publications, MDPI, vol. 9(1), pages 1-12, March.
    7. Cheng, Xi & Wang, Haoran & Tang, Li & Jiang, Weiyan & Zhou, Maotian & Wang, Guoyan, 2024. "Open peer review correlates with altmetrics but not with citations: Evidence from Nature Communications and PLoS One," Journal of Informetrics, Elsevier, vol. 18(3).
    8. ederico Bianchi & Flaminio Squazzoni, 2022. "Can transparency undermine peer review? A simulation model of scientist behavior under open peer review [Reviewing Peer Review]," Science and Public Policy, Oxford University Press, vol. 49(5), pages 791-800.
    9. Xiaobo Tang & Xin Du & Qiongfu Wang & Jialin Wu, 2024. "Intelligent recognition of high-quality academic papers: based on knowledge-based metasemantic networks," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(11), pages 6779-6812, November.
    10. Hou, Li & Wu, Qiang & Xie, Yundong, 2024. "Does open identity of peer reviewers positively relate to citations?," Journal of Informetrics, Elsevier, vol. 18(1).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kritana Prueksakorn & Cheng-Xu Piao & Hyunchul Ha & Taehyeung Kim, 2015. "Computational and Experimental Investigation for an Optimal Design of Industrial Windows to Allow Natural Ventilation during Wind-Driven Rain," Sustainability, MDPI, vol. 7(8), pages 1-22, August.
    2. Hualin Xie & Jinlang Zou & Hailing Jiang & Ning Zhang & Yongrok Choi, 2014. "Spatiotemporal Pattern and Driving Forces of Arable Land-Use Intensity in China: Toward Sustainable Land Management Using Emergy Analysis," Sustainability, MDPI, vol. 6(6), pages 1-17, May.
    3. Stephan E. Maurer & Andrei V. Potlogea, 2021. "Male‐biased Demand Shocks and Women's Labour Force Participation: Evidence from Large Oil Field Discoveries," Economica, London School of Economics and Political Science, vol. 88(349), pages 167-188, January.
    4. Tie Hua Zhou & Ling Wang & Keun Ho Ryu, 2015. "Supporting Keyword Search for Image Retrieval with Integration of Probabilistic Annotation," Sustainability, MDPI, vol. 7(5), pages 1-18, May.
    5. T. Karski, 2019. "Opinions and Controversies in Problem of The So-Called Idiopathic Scoliosis. Information About Etiology, New Classification and New Therapy," Biomedical Journal of Scientific & Technical Research, Biomedical Research Network+, LLC, vol. 12(5), pages 9612-9616, January.
    6. Wesley Mendes-da-Silva, 2020. "What Makes an Article be More Cited?," RAC - Revista de Administração Contemporânea (Journal of Contemporary Administration), ANPAD - Associação Nacional de Pós-Graduação e Pesquisa em Administração, vol. 24(6), pages 507-513.
    7. Wisdom Akpalu & Mintewab Bezabih, 2015. "Tenure Insecurity, Climate Variability and Renting out Decisions among Female Small-Holder Farmers in Ethiopia," Sustainability, MDPI, vol. 7(6), pages 1-16, June.
    8. Wei Chen & Shu-Yu Liu & Chih-Han Chen & Yi-Shan Lee, 2011. "Bounded Memory, Inertia, Sampling and Weighting Model for Market Entry Games," Games, MDPI, vol. 2(1), pages 1-13, March.
    9. David Harborth & Sebastian Pape, 2020. "Empirically Investigating Extraneous Influences on the “APCO” Model—Childhood Brand Nostalgia and the Positivity Bias," Future Internet, MDPI, vol. 12(12), pages 1-16, December.
    10. Taeyeoun Roh & Yujin Jeong & Byungun Yoon, 2017. "Developing a Methodology of Structuring and Layering Technological Information in Patent Documents through Natural Language Processing," Sustainability, MDPI, vol. 9(11), pages 1-19, November.
    11. He-Yau Kang & Amy H. I. Lee & Tzu-Ting Huang, 2016. "Project Management for a Wind Turbine Construction by Applying Fuzzy Multiple Objective Linear Programming Models," Energies, MDPI, vol. 9(12), pages 1-15, December.
    12. A. B. Atkinson & Stephen P. Jenkins, 2020. "A Different Perspective on the Evolution of UK Income Inequality," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 66(2), pages 253-266, June.
    13. Haiyan Xu & Yanhui Ding & Jing Sun & Kun Zhao & Yuanjian Chen, 2019. "Dynamic Group Recommendation Based on the Attention Mechanism," Future Internet, MDPI, vol. 11(9), pages 1-15, September.
    14. Adina Letiţia Negruşa & Valentin Toader & Aurelian Sofică & Mihaela Filofteia Tutunea & Rozalia Veronica Rus, 2015. "Exploring Gamification Techniques and Applications for Sustainable Tourism," Sustainability, MDPI, vol. 7(8), pages 1-30, August.
    15. Ahmad N. Alkenani & Mohammad Ashraf & Ghulam Mohammad, 2020. "Quantum Codes from Constacyclic Codes over the Ring F q [ u 1 , u 2 ]/〈 u 1 2 - u 1 , u 2 2 - u 2 , u 1 u 2 - u 2 u 1 〉," Mathematics, MDPI, vol. 8(5), pages 1-11, May.
    16. Shang-Yuan Chen & Jui-Ting Huang, 2012. "A Smart Green Building: An Environmental Health Control Design," Energies, MDPI, vol. 5(5), pages 1-16, May.
    17. Yanhong Feng & Xu Yu & Gai-Ge Wang, 2019. "A Novel Monarch Butterfly Optimization with Global Position Updating Operator for Large-Scale 0-1 Knapsack Problems," Mathematics, MDPI, vol. 7(11), pages 1-31, November.
    18. Xiaoshu Cao & Feiwen Liang & Huiling Chen & Yongwei Liu, 2017. "Circuity Characteristics of Urban Travel Based on GPS Data: A Case Study of Guangzhou," Sustainability, MDPI, vol. 9(11), pages 1-21, November.
    19. S. B. Reshetnikov & M. R. Skirdov, 2017. "Analysis of methodological approaches to determination and assessment of the human capital," Russian Journal of Industrial Economics, MISIS, vol. 10(1).
    20. Mi Jung Son & Jin Han Park & Ka Hyun Ko, 2019. "Some Hesitant Fuzzy Hamacher Power-Aggregation Operators for Multiple-Attribute Decision-Making," Mathematics, MDPI, vol. 7(7), pages 1-33, July.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:125:y:2020:i:2:d:10.1007_s11192-020-03488-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.