IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v85y2010i3d10.1007_s11192-010-0215-7.html
   My bibliography  Save this article

The validity of staff editors’ initial evaluations of manuscripts: a case study of Angewandte Chemie International Edition

Author

Listed:
  • Lutz Bornmann

    (ETH Zurich)

  • Hans-Dieter Daniel

    (ETH Zurich
    University of Zurich, Evaluation Office)

Abstract

This paper investigates the extent to which staff editors’ evaluations of submitted manuscripts—that is, internal evaluations carried out before external peer reviewing—are valid. To answer this question we utilized data on the manuscript reviewing process at the journal Angewandte Chemie International Edition. The results of this study indicate that the initial internal evaluations are valid. Further, it appears that external review is indispensable for the decision on the publication worthiness of manuscripts: (1) For the majority of submitted manuscripts, staff editors are uncertain about publication worthiness; (2) there is a statistically significant proportional difference in “Rejection” between the editors' initial evaluation and the final editorial decision (after peer review); (3) three-quarters of the manuscripts that were rated negatively at the initial internal evaluation but accepted for publication after the peer review had far above-average citation counts.

Suggested Citation

  • Lutz Bornmann & Hans-Dieter Daniel, 2010. "The validity of staff editors’ initial evaluations of manuscripts: a case study of Angewandte Chemie International Edition," Scientometrics, Springer;Akadémiai Kiadó, vol. 85(3), pages 681-687, December.
    • Handle: RePEc:spr:scient:v:85:y:2010:i:3:d:10.1007_s11192-010-0215-7
    DOI: 10.1007/s11192-010-0215-7
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-010-0215-7
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-010-0215-7?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Quinn McNemar, 1947. "Note on the sampling error of the difference between correlated proportions or percentages," Psychometrika, Springer;The Psychometric Society, vol. 12(2), pages 153-157, June.
    2. Craig, Iain D. & Plume, Andrew M. & McVeigh, Marie E. & Pringle, James & Amin, Mayur, 2007. "Do open access articles have greater citation impact?," Journal of Informetrics, Elsevier, vol. 1(3), pages 239-248.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Azzurra Ragone & Katsiaryna Mirylenka & Fabio Casati & Maurizio Marchese, 2013. "On peer review in computer science: analysis of its effectiveness and suggestions for improvement," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(2), pages 317-356, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Uttam Bandyopadhyay & Atanu Biswas & Shirsendu Mukherjee, 2009. "Adaptive two-treatment two-period crossover design for binary treatment responses incorporating carry-over effects," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 18(1), pages 13-33, March.
    2. Mark J. McCabe & Christopher M. Snyder, 2015. "Does Online Availability Increase Citations? Theory and Evidence from a Panel of Economics and Business Journals," The Review of Economics and Statistics, MIT Press, vol. 97(1), pages 144-165, March.
    3. Preety Srivastava & Xueyan Zhao, 2010. "What Do the Bingers Drink? Micro‐Unit Evidence on Negative Externalities and Drinker Characteristics of Alcohol Consumption by Beverage Types," Economic Papers, The Economic Society of Australia, vol. 29(2), pages 229-250, June.
    4. Holger Schwender & Margaret A. Taub & Terri H. Beaty & Mary L. Marazita & Ingo Ruczinski, 2012. "Rapid Testing of SNPs and Gene–Environment Interactions in Case–Parent Trio Data Based on Exact Analytic Parameter Estimation," Biometrics, The International Biometric Society, vol. 68(3), pages 766-773, September.
    5. Matysková, Ludmila & Rogers, Brian & Steiner, Jakub & Sun, Keh-Kuan, 2020. "Habits as adaptations: An experimental study," Games and Economic Behavior, Elsevier, vol. 122(C), pages 391-406.
    6. André, Kévin, 2013. "Applying the Capability Approach to the French Education System: An Assessment of the "Pourquoi pas moi ?"," ESSEC Working Papers WP1316, ESSEC Research Center, ESSEC Business School.
    7. Hajar Sotudeh & Zahra Ghasempour & Maryam Yaghtin, 2015. "The citation advantage of author-pays model: the case of Springer and Elsevier OA journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 104(2), pages 581-608, August.
    8. Ruiz-Frau, A. & Krause, T. & Marbà , N., 2018. "The use of sociocultural valuation in sustainable environmental management," Ecosystem Services, Elsevier, vol. 29(PA), pages 158-167.
    9. Juliana Loureiro Almeida Campos & André Sobral & Josivan Soares Silva & Thiago Antonio Sousa Araújo & Washington Soares Ferreira-Júnior & Flávia Rosa Santoro & Gilney Charll Santos & Ulysses Paulino A, 2016. "Insularity and citation behavior of scientific articles in young fields: the case of ethnobiology," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 1037-1055, November.
    10. Wang, Zhiqi & Chen, Yue & Glänzel, Wolfgang, 2020. "Preprints as accelerator of scholarly communication: An empirical analysis in Mathematics," Journal of Informetrics, Elsevier, vol. 14(4).
    11. AlMalki, Hameeda A. & Durugbo, Christopher M., 2023. "Evaluating critical institutional factors of Industry 4.0 for education reform," Technological Forecasting and Social Change, Elsevier, vol. 188(C).
    12. Chi, Yuxue & Tang, Xianyi & Liu, Yijun, 2022. "Exploring the “awakening effect” in knowledge diffusion: a case study of publications in the library and information science domain," Journal of Informetrics, Elsevier, vol. 16(4).
    13. Guevara, C. Angelo & Fukushi, Mitsuyoshi, 2016. "Modeling the decoy effect with context-RUM Models: Diagrammatic analysis and empirical evidence from route choice SP and mode choice RP case studies," Transportation Research Part B: Methodological, Elsevier, vol. 93(PA), pages 318-337.
    14. Alexandra I. Khalyasmaa & Pavel V. Matrenin & Stanislav A. Eroshenko & Vadim Z. Manusov & Andrey M. Bramm & Alexey M. Romanov, 2022. "Data Mining Applied to Decision Support Systems for Power Transformers’ Health Diagnostics," Mathematics, MDPI, vol. 10(14), pages 1-25, July.
    15. Arnaldo Rabello de Aguiar Vallim Filho & Daniel Farina Moraes & Marco Vinicius Bhering de Aguiar Vallim & Leilton Santos da Silva & Leandro Augusto da Silva, 2022. "A Machine Learning Modeling Framework for Predictive Maintenance Based on Equipment Load Cycle: An Application in a Real World Case," Energies, MDPI, vol. 15(10), pages 1-41, May.
    16. Alireza Taheri Dehkordi & Mohammad Javad Valadan Zoej & Hani Ghasemi & Ebrahim Ghaderpour & Quazi K. Hassan, 2022. "A New Clustering Method to Generate Training Samples for Supervised Monitoring of Long-Term Water Surface Dynamics Using Landsat Data through Google Earth Engine," Sustainability, MDPI, vol. 14(13), pages 1-24, June.
    17. Eberhard Feess & Marc Scheufen, 2016. "Academic copyright in the publishing game: a contest perspective," European Journal of Law and Economics, Springer, vol. 42(2), pages 263-294, October.
    18. Ennas, Gianfranco & Di Guardo, Maria Chiara, 2015. "Features of top-rated gold open access journals: An analysis of the scopus database," Journal of Informetrics, Elsevier, vol. 9(1), pages 79-89.
    19. Bornmann, Lutz & Daniel, Hans-Dieter, 2009. "Extent of type I and type II errors in editorial decisions: A case study on Angewandte Chemie International Edition," Journal of Informetrics, Elsevier, vol. 3(4), pages 348-352.
    20. Lahtinen, Tuomas J. & Hämäläinen, Raimo P., 2016. "Path dependence and biases in the even swaps decision analysis method," European Journal of Operational Research, Elsevier, vol. 249(3), pages 890-898.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:85:y:2010:i:3:d:10.1007_s11192-010-0215-7. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.