IDEAS home Printed from https://ideas.repec.org/a/oup/scippl/v49y2022i5p791-800..html
   My bibliography  Save this article

Can transparency undermine peer review? A simulation model of scientist behavior under open peer review
[Reviewing Peer Review]

Author

Listed:
  • ederico Bianchi
  • Flaminio Squazzoni

Abstract

Transparency and accountability are keywords in corporate business, politics, and science. As part of the open science movement, many journals have started to adopt forms of open peer review beyond the closed (single- or double-blind) standard model. However, there is contrasting evidence on the impact of these innovations on the quality of peer review. Furthermore, their long-term consequences on scientists’ cooperation and competition are difficult to assess empirically. This paper aims to fill this gap by presenting an agent-based model that simulates competition and status dynamics between scholars in an artificial academic system. The results would suggest that if referees are sensitive to competition and status, the transparency achieved by open peer review could backfire on the quality of the process. Although only abstract and hypothetical, our findings suggest the importance of multidimensional values of peer review and the anonymity and confidentiality of the process.

Suggested Citation

  • ederico Bianchi & Flaminio Squazzoni, 2022. "Can transparency undermine peer review? A simulation model of scientist behavior under open peer review [Reviewing Peer Review]," Science and Public Policy, Oxford University Press, vol. 49(5), pages 791-800.
    • Handle: RePEc:oup:scippl:v:49:y:2022:i:5:p:791-800.
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1093/scipol/scac027
    Download Restriction: Access to full text is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Dietmar Wolfram & Peiling Wang & Adam Hembree & Hyoungjoo Park, 2020. "Open peer review: promoting transparency in open science," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 1033-1051, November.
    2. Oecd, 2008. "DAC Peer Review of Greece," OECD Journal on Development, OECD Publishing, vol. 7(4), pages 7-93.
    3. Thomas Feliciani & Junwen Luo & Lai Ma & Pablo Lucas & Flaminio Squazzoni & Ana Marušić & Kalpana Shankar, 2019. "A scoping review of simulation models of peer review," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(1), pages 555-594, October.
    4. Oecd, 2008. "DAC Peer Review of the United Kingdom," OECD Journal on Development, OECD Publishing, vol. 7(3), pages 7-114.
    5. Oecd, 2008. "DAC Peer Review of the Netherlands," OECD Journal on Development, OECD Publishing, vol. 7(3), pages 115-214.
    6. Giangiacomo Bravo & Francisco Grimaldo & Emilia López-Iñesta & Bahar Mehmani & Flaminio Squazzoni, 2019. "The effect of publishing peer review reports on referee behavior in five scholarly journals," Nature Communications, Nature, vol. 10(1), pages 1-8, December.
    7. Oecd, 2008. "DAC Peer Review of Denmark," OECD Journal on Development, OECD Publishing, vol. 8(4), pages 7-125.
    8. Teplitskiy, Misha & Acuna, Daniel & Elamrani-Raoult, Aïda & Körding, Konrad & Evans, James, 2018. "The sociology of scientific validity: How professional networks shape judgement in peer review," Research Policy, Elsevier, vol. 47(9), pages 1825-1841.
    9. Oecd, 2008. "DAC Peer Review of the European Community," OECD Journal on Development, OECD Publishing, vol. 8(4), pages 127-261.
    10. Ben R. Martin, 2016. "Twenty challenges for innovation studies," Science and Public Policy, Oxford University Press, vol. 43(3), pages 432-450.
    11. Flaminio Squazzoni & Petra Ahrweiler & Tiago Barros & Federico Bianchi & Aliaksandr Birukou & Harry J. J. Blom & Giangiacomo Bravo & Stephen Cowley & Virginia Dignum & Pierpaolo Dondio & Francisco Gri, 2020. "Unlock ways to share data on peer review," Nature, Nature, vol. 578(7796), pages 512-514, February.
    12. Michail Kovanis & Raphaël Porcher & Philippe Ravaud & Ludovic Trinquart, 2016. "The Global Burden of Journal Peer Review in the Biomedical Literature: Strong Imbalance in the Collective Enterprise," PLOS ONE, Public Library of Science, vol. 11(11), pages 1-14, November.
    13. Syavash Nobarany & Kellogg S. Booth, 2017. "Understanding and supporting anonymity policies in peer review," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 68(4), pages 957-971, April.
    14. JinHyo Joseph Yun & DongKyu Won & ByungYong Hwang & JinWon Kang & DongHwan Kim, 2015. "Analysing and simulating the effects of open innovation policies: Application of the results to Cambodia," Science and Public Policy, Oxford University Press, vol. 42(6), pages 743-760.
    15. Flaminio Squazzoni & Claudio Gandelli, 2013. "Opening the Black-Box of Peer Review: An Agent-Based Model of Scientist Behaviour," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 16(2), pages 1-3.
    16. Emre Sarigöl & David Garcia & Ingo Scholtes & Frank Schweitzer, 2017. "Quantifying the effect of editor–author relations on manuscript handling times," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 609-631, October.
    17. Nigel Gilbert & Petra Ahrweiler & Pete Barbrook-Johnson & Kavin Preethi Narasimhan & Helen Wilkinson, 2018. "Computational Modelling of Public Policy: Reflections on Practice," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 21(1), pages 1-14.
    18. Bruce Edmonds & Christophe Le Page & Mike Bithell & Edmund Chattoe-Brown & Volker Grimm & Ruth Meyer & Cristina Montañola-Sales & Paul Ormerod & Hilton Root & Flaminio Squazzoni, 2019. "Different Modelling Purposes," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 22(3), pages 1-6.
    19. Federico Bianchi & Francisco Grimaldo & Giangiacomo Bravo & Flaminio Squazzoni, 2018. "The peer review game: an agent-based model of scientists facing resource constraints and institutional pressures," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 1401-1420, September.
    20. Oecd, 2008. "DAC Peer Review of Canada," OECD Journal on Development, OECD Publishing, vol. 8(4), pages 263-387.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Sun, Zhuanlan & Clark Cao, C. & Ma, Chao & Li, Yiwei, 2023. "The academic status of reviewers predicts their language use," Journal of Informetrics, Elsevier, vol. 17(4).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sun, Zhuanlan & Clark Cao, C. & Ma, Chao & Li, Yiwei, 2023. "The academic status of reviewers predicts their language use," Journal of Informetrics, Elsevier, vol. 17(4).
    2. Maciej J. Mrowinski & Agata Fronczak & Piotr Fronczak & Olgica Nedic & Aleksandar Dekanski, 2020. "The hurdles of academic publishing from the perspective of journal editors: a case study," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 115-133, October.
    3. Zhao, Zhi-Dan & Chen, Jiahao & Lu, Yichuan & Zhao, Na & Jiang, Dazhi & Wang, Bing-Hong, 2021. "Dynamic patterns of open review process," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 582(C).
    4. Besim Bilalli & Rana Faisal Munir & Alberto Abelló, 2021. "A framework for assessing the peer review duration of journals: case study in computer science," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(1), pages 545-563, January.
    5. Siddarth Srinivasan & Jamie Morgenstern, 2021. "Auctions and Peer Prediction for Academic Peer Review," Papers 2109.00923, arXiv.org, revised May 2023.
    6. Stephen A Gallo & Afton S Carpenter & Scott R Glisson, 2013. "Teleconference versus Face-to-Face Scientific Peer Review of Grant Application: Effects on Review Outcomes," PLOS ONE, Public Library of Science, vol. 8(8), pages 1-9, August.
    7. Feliciani, Thomas & Morreau, Michael & Luo, Junwen & Lucas, Pablo & Shankar, Kalpana, 2022. "Designing grant-review panels for better funding decisions: Lessons from an empirically calibrated simulation model," Research Policy, Elsevier, vol. 51(4).
    8. José Cendejas Bueno & Cecilia Font de Villanueva, 2015. "Convergence of inflation with a common cycle: estimating and modelling Spanish historical inflation from the 16th to the 18th centuries," Empirical Economics, Springer, vol. 48(4), pages 1643-1665, June.
    9. Lucas Rodriguez Forti & Luiz A. Solino & Judit K. Szabo, 2021. "Trade-off between urgency and reduced editorial capacity affect publication speed in ecological and medical journals during 2020," Palgrave Communications, Palgrave Macmillan, vol. 8(1), pages 1-9, December.
    10. Bravo, Giangiacomo & Farjam, Mike & Grimaldo Moreno, Francisco & Birukou, Aliaksandr & Squazzoni, Flaminio, 2018. "Hidden connections: Network effects on editorial decisions in four computer science journals," Journal of Informetrics, Elsevier, vol. 12(1), pages 101-112.
    11. Zhang, Guangyao & Xu, Shenmeng & Sun, Yao & Jiang, Chunlin & Wang, Xianwen, 2022. "Understanding the peer review endeavor in scientific publishing," Journal of Informetrics, Elsevier, vol. 16(2).
    12. Federico Bianchi & Francisco Grimaldo & Giangiacomo Bravo & Flaminio Squazzoni, 2018. "The peer review game: an agent-based model of scientists facing resource constraints and institutional pressures," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 1401-1420, September.
    13. J. A. Garcia & Rosa Rodriguez-Sánchez & J. Fdez-Valdivia, 2021. "The interplay between the reviewer’s incentives and the journal’s quality standard," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(4), pages 3041-3061, April.
    14. Louise Bedsworth, 2012. "California’s local health agencies and the state’s climate adaptation strategy," Climatic Change, Springer, vol. 111(1), pages 119-133, March.
    15. Andreas Ortmann & Benoît Walraevens, 2012. "Adam Smith, Philosopher and Man of the World," Post-Print halshs-00756341, HAL.
    16. Bianchi, Federico & Grimaldo, Francisco & Squazzoni, Flaminio, 2019. "The F3-index. Valuing reviewers for scholarly journals," Journal of Informetrics, Elsevier, vol. 13(1), pages 78-86.
    17. Lokman Tutuncu & Recep Yucedogru & Idris Sarisoy, 2022. "Academic favoritism at work: insider bias in Turkish national journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(5), pages 2547-2576, May.
    18. Thomas Feliciani & Junwen Luo & Lai Ma & Pablo Lucas & Flaminio Squazzoni & Ana Marušić & Kalpana Shankar, 2019. "A scoping review of simulation models of peer review," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(1), pages 555-594, October.
    19. J. A. Garcia & Rosa Rodriguez-Sánchez & J. Fdez-Valdivia, 2020. "The author–reviewer game," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(3), pages 2409-2431, September.
    20. Akbaritabar, Aliakbar & Stephen, Dimity & Squazzoni, Flaminio, 2022. "A study of referencing changes in preprint-publication pairs across multiple fields," Journal of Informetrics, Elsevier, vol. 16(2).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:oup:scippl:v:49:y:2022:i:5:p:791-800.. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Oxford University Press (email available below). General contact details of provider: https://academic.oup.com/spp .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.