IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0013345.html
   My bibliography  Save this article

Do Author-Suggested Reviewers Rate Submissions More Favorably than Editor-Suggested Reviewers? A Study on Atmospheric Chemistry and Physics

Author

Listed:
  • Lutz Bornmann
  • Hans-Dieter Daniel

Abstract

Background: Ratings in journal peer review can be affected by sources of bias. The bias variable investigated here was the information on whether authors had suggested a possible reviewer for their manuscript, and whether the editor had taken up that suggestion or had chosen a reviewer that had not been suggested by the authors. Studies have shown that author-suggested reviewers rate manuscripts more favorably than editor-suggested reviewers do. Methodology/Principal Findings: Reviewers' ratings on three evaluation criteria and the reviewers' final publication recommendations were available for 552 manuscripts (in total 1145 reviews) that were submitted to Atmospheric Chemistry and Physics, an interactive open access journal using public peer review (authors' and reviewers' comments are publicly exchanged). Public peer review is supposed to bring a new openness to the reviewing process that will enhance its objectivity. In the statistical analysis the quality of a manuscript was controlled for to prevent favorable reviewers' ratings from being attributable to quality instead of to the bias variable. Conclusions/Significance: Our results agree with those from other studies that editor-suggested reviewers rated manuscripts between 30% and 42% less favorably than author-suggested reviewers. Against this backdrop journal editors should consider either doing without the use of author-suggested reviewers or, if they are used, bringing in more than one editor-suggested reviewer for the review process (so that the review by author-suggested reviewers can be put in perspective).

Suggested Citation

  • Lutz Bornmann & Hans-Dieter Daniel, 2010. "Do Author-Suggested Reviewers Rate Submissions More Favorably than Editor-Suggested Reviewers? A Study on Atmospheric Chemistry and Physics," PLOS ONE, Public Library of Science, vol. 5(10), pages 1-8, October.
  • Handle: RePEc:plo:pone00:0013345
    DOI: 10.1371/journal.pone.0013345
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0013345
    Download Restriction: no

    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0013345&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pone.0013345?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Craig, Iain D. & Plume, Andrew M. & McVeigh, Marie E. & Pringle, James & Amin, Mayur, 2007. "Do open access articles have greater citation impact?," Journal of Informetrics, Elsevier, vol. 1(3), pages 239-248.
    2. Laband, David N & Piette, Michael J, 1994. "Favoritism versus Search for Good Papers: Empirical Evidence Regarding the Behavior of Journal Editors," Journal of Political Economy, University of Chicago Press, vol. 102(1), pages 194-203, February.
    3. J. Scott Long & Jeremy Freese, 2006. "Regression Models for Categorical Dependent Variables using Stata, 2nd Edition," Stata Press books, StataCorp LLC, edition 2, number long2, January.
    4. Bornmann, Lutz & Mutz, Rüdiger & Daniel, Hans-Dieter, 2008. "How to detect indications of potential sources of bias in peer review: A generalized latent variable modeling approach exemplified by a gender study," Journal of Informetrics, Elsevier, vol. 2(4), pages 280-287.
    5. Lutz Bornmann & Hans-Dieter Daniel, 2009. "Reviewer and editor biases in journal peer review: an investigation of manuscript refereeing at Angewandte Chemie International Edition," Research Evaluation, Oxford University Press, vol. 18(4), pages 262-272, October.
    6. Upali W. Jayasinghe & Herbert W. Marsh & Nigel Bond, 2003. "A multilevel cross‐classified modelling approach to peer review of grant proposals: the effects of assessor and researcher attributes on assessor ratings," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 166(3), pages 279-300, October.
    7. Austin Nichols, 2007. "Causal inference with observational data," Stata Journal, StataCorp LLC, vol. 7(4), pages 507-541, December.
    8. Lutz Bornmann & Werner Marx & Hermann Schier & Andreas Thor & Hans-Dieter Daniel, 2010. "From black box to white box at open access journals: predictive validity of manuscript reviewing and editorial decisions at Atmospheric Chemistry and Physics," Research Evaluation, Oxford University Press, vol. 19(2), pages 105-118, June.
    9. Scott Smart & Joel Waldfogel, 1996. "A Citation-Based Test for Discrimination at Economics and Finance Journals," NBER Working Papers 5460, National Bureau of Economic Research, Inc.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Carole J. Lee & Cassidy R. Sugimoto & Guo Zhang & Blaise Cronin, 2013. "Bias in peer review," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(1), pages 2-17, January.
    2. Cassidy R. Sugimoto & Blaise Cronin, 2013. "Citation gamesmanship: testing for evidence of ego bias in peer review," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(3), pages 851-862, June.
    3. Lawson, Nicholas, 2023. "What citation tests really tell us about bias in academic publishing," European Economic Review, Elsevier, vol. 158(C).
    4. Daniel S. Hamermesh, 2018. "Citations in Economics: Measurement, Uses, and Impacts," Journal of Economic Literature, American Economic Association, vol. 56(1), pages 115-156, March.
    5. Ennas, Gianfranco & Di Guardo, Maria Chiara, 2015. "Features of top-rated gold open access journals: An analysis of the scopus database," Journal of Informetrics, Elsevier, vol. 9(1), pages 79-89.
    6. Philip R P Coelho & James E McClure & Peter J Reilly, 2014. "An Investigation of Editorial Favoritism in the AER," Eastern Economic Journal, Palgrave Macmillan;Eastern Economic Association, vol. 40(2), pages 274-281, March.
    7. David Card & Stefano DellaVigna, 2020. "What Do Editors Maximize? Evidence from Four Economics Journals," The Review of Economics and Statistics, MIT Press, vol. 102(1), pages 195-217, March.
    8. David Michayluk & Ralf Zurbruegg, 2014. "Do lead articles signal higher quality in the digital age? Evidence from finance journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(2), pages 961-973, February.
    9. Tang, Li & Yang, Defang & Wang, Mingxing & Guo, Ying, 2024. "The mediating impact of citation scope: Evidence from China's ESI publications," Journal of Informetrics, Elsevier, vol. 18(3).
    10. Teplitskiy, Misha & Acuna, Daniel & Elamrani-Raoult, Aïda & Körding, Konrad & Evans, James, 2018. "The sociology of scientific validity: How professional networks shape judgement in peer review," Research Policy, Elsevier, vol. 47(9), pages 1825-1841.
    11. Andreas Nishikawa-Pacher & Tamara Heck & Kerstin Schoch, 2023. "Open Editors: A dataset of scholarly journals’ editorial board positions," Research Evaluation, Oxford University Press, vol. 32(2), pages 228-243.
    12. Squazzoni, Flaminio & Gandelli, Claudio, 2012. "Saint Matthew strikes again: An agent-based model of peer review and the scientific community structure," Journal of Informetrics, Elsevier, vol. 6(2), pages 265-275.
    13. Si, Kao & Li, Yiwei & Ma, Chao & Guo, Feng, 2023. "Affiliation bias in peer review and the gender gap," Research Policy, Elsevier, vol. 52(7).
    14. Jinsuk Yang & Qing Hao & Mahmut Yaşar, 2023. "Institutional investors and cross‐border mergers and acquisitions: The 2000–2018 period," International Review of Finance, International Review of Finance Ltd., vol. 23(3), pages 553-583, September.
    15. Gregory Thompson & Jeffrey Brown & Torsha Bhattacharya, 2012. "What Really Matters for Increasing Transit Ridership: Understanding the Determinants of Transit Ridership Demand in Broward County, Florida," Urban Studies, Urban Studies Journal Limited, vol. 49(15), pages 3327-3345, November.
    16. repec:rza:wpaper:227 is not listed on IDEAS
    17. Seung-Whan Choi & James A. Piazza, 2017. "Foreign Military Interventions and Suicide Attacks," Journal of Conflict Resolution, Peace Science Society (International), vol. 61(2), pages 271-297, February.
    18. Clara Berridge & Yuanjin Zhou & Julie M. Robillard & Jeffrey Kaye, 2023. "AI Companion Robot Data Sharing: Preferences of an Online Cohort and Policy Implications," Journal of Elder Policy, John Wiley & Sons, vol. 2(3), pages 19-54, June.
    19. Mikael Svensson & Fredrik Nilsson & Karl Arnberg, 2015. "Reimbursement Decisions for Pharmaceuticals in Sweden: The Impact of Disease Severity and Cost Effectiveness," PharmacoEconomics, Springer, vol. 33(11), pages 1229-1236, November.
    20. Aurelia Magdalena Pisoschi & Claudia Gabriela Pisoschi, 2016. "Is open access the solution to increase the impact of scientific journals?," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 1075-1095, November.
    21. Dixon, Huw D. & Grimme, Christian, 2022. "State-dependent or time-dependent pricing? New evidence from a monthly firm-level survey: 1980–2017," European Economic Review, Elsevier, vol. 150(C).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0013345. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.