IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v126y2021i7d10.1007_s11192-021-04032-8.html
   My bibliography  Save this article

Understanding authors' psychological reactions to peer reviews: a text mining approach

Author

Listed:
  • Shan Jiang

    (University of Massachusetts Boston)

Abstract

Peer reviews play a vital role in academic publishing. Authors have various feelings towards peer reviews. This study analyzes the experiences shared by authors in Scirev.org to understand these authors' psychological reactions to several aspects of peer reviews, including decisions, turnaround time, the number of reviews, and review quality. Text mining was used to extract different types of psychological reactions of authors, including affective processes and cognitive processes. Results show that authors' psychological responses to peer reviews are complex and cannot be summarized by a single numerical rating directly given by the authors. Rejection invokes anger, sadness, and disagreement, but not anxiety. Fast turnaround arouses positive emotions from authors, but slow peer review processes do not increase negative emotions as much. Low-quality reviews lead to a wide array of negative emotions, including anxiety, anger, and sadness.

Suggested Citation

  • Shan Jiang, 2021. "Understanding authors' psychological reactions to peer reviews: a text mining approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 6085-6103, July.
    • Handle: RePEc:spr:scient:v:126:y:2021:i:7:d:10.1007_s11192-021-04032-8
    DOI: 10.1007/s11192-021-04032-8
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-021-04032-8
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-021-04032-8?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Niccolò Casnici & Francisco Grimaldo & Nigel Gilbert & Pierpaolo Dondio & Flaminio Squazzoni, 2017. "Assessing peer review by gauging the fate of rejected manuscripts: the case of the Journal of Artificial Societies and Social Simulation," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 533-546, October.
    2. Ashlee Humphreys & Rebecca Jen-Hui Wang & Eileen FischerEditor & Linda PriceAssociate Editor, 2018. "Automated Text Analysis for Consumer Research," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 44(6), pages 1274-1306.
    3. Lutz Bornmann & Markus Wolf & Hans-Dieter Daniel, 2012. "Closed versus open reviewing of journal manuscripts: how far do comments differ in language use?," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(3), pages 843-856, June.
    4. David J. Solomon & Bo‐Christer Björk, 2012. "Publication fees in open access publishing: Sources of funding and factors influencing choice of journal," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 63(1), pages 98-107, January.
    5. Rex Dalton, 2001. "Peers under pressure," Nature, Nature, vol. 413(6852), pages 102-104, September.
    6. Kübler, Raoul V. & Colicev, Anatoli & Pauwels, Koen H., 2020. "Social Media's Impact on the Consumer Mindset: When to Use Which Sentiment Extraction Tool?," Journal of Interactive Marketing, Elsevier, vol. 50(C), pages 136-155.
    7. Janine Huisman & Jeroen Smits, 2017. "Duration and quality of the peer review process: the author’s perspective," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 633-650, October.
    8. J. Rigby & D. Cox & K. Julian, 2018. "Journal peer review: a bar or bridge? An analysis of a paper’s revision history and turnaround time, and the effect on citation," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 1087-1105, March.
    9. Peter van den Besselaar & Ulf Sandström & Hélène Schiffbaenker, 2018. "Studying grant decision-making: a linguistic analysis of review reports," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(1), pages 313-329, October.
    10. Bernd Pulverer, 2010. "Transparency showcases strength of peer review," Nature, Nature, vol. 468(7320), pages 29-31, November.
    11. David J. Solomon & Bo-Christer Björk, 2012. "Publication fees in open access publishing: Sources of funding and factors influencing choice of journal," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(1), pages 98-107, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Sun, Zhuanlan, 2024. "Textual features of peer review predict top-cited papers: An interpretable machine learning perspective," Journal of Informetrics, Elsevier, vol. 18(2).
    2. Iván Aranzales & Ho Fai Chan & Benno Torgler, 2023. "Finally! How time lapse in Nobel Prize reception affects emotionality in the Nobel Prize banquet speeches," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(7), pages 4089-4115, July.
    3. Sun, Zhuanlan & Clark Cao, C. & Ma, Chao & Li, Yiwei, 2023. "The academic status of reviewers predicts their language use," Journal of Informetrics, Elsevier, vol. 17(4).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Katrin Hussinger & Lorenzo Palladini, 2024. "Information accessibility and knowledge creation: the impact of Google’s withdrawal from China on scientific research," Industry and Innovation, Taylor & Francis Journals, vol. 31(6), pages 753-783, July.
    2. Sun, Zhuanlan, 2024. "Textual features of peer review predict top-cited papers: An interpretable machine learning perspective," Journal of Informetrics, Elsevier, vol. 18(2).
    3. Lokman Tutuncu, 2023. "All-pervading insider bias alters review time in Turkish university journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(6), pages 3743-3791, June.
    4. Besim Bilalli & Rana Faisal Munir & Alberto Abelló, 2021. "A framework for assessing the peer review duration of journals: case study in computer science," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(1), pages 545-563, January.
    5. Hren, Darko & Pina, David G. & Norman, Christopher R. & Marušić, Ana, 2022. "What makes or breaks competitive research proposals? A mixed-methods analysis of research grant evaluation reports," Journal of Informetrics, Elsevier, vol. 16(2).
    6. Stuart Lawson, 2015. "Fee Waivers for Open Access Journals," Publications, MDPI, vol. 3(3), pages 1-13, August.
    7. Balázs Győrffy & Andrea Magda Nagy & Péter Herman & Ádám Török, 2018. "Factors influencing the scientific performance of Momentum grant holders: an evaluation of the first 117 research groups," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(1), pages 409-426, October.
    8. Ana Teresa Santos & Sandro Mendonça, 2022. "Do papers (really) match journals’ “aims and scope”? A computational assessment of innovation studies," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 7449-7470, December.
    9. Jørgen Burchardt, 2014. "Researchers Outside APC-Financed Open Access," SAGE Open, , vol. 4(4), pages 21582440145, September.
    10. Zhuanlan Sun & C. Clark Cao & Sheng Liu & Yiwei Li & Chao Ma, 2024. "Behavioral consequences of second-person pronouns in written communications between authors and reviewers of scientific papers," Nature Communications, Nature, vol. 15(1), pages 1-12, December.
    11. Bianchi, Federico & Grimaldo, Francisco & Squazzoni, Flaminio, 2019. "The F3-index. Valuing reviewers for scholarly journals," Journal of Informetrics, Elsevier, vol. 13(1), pages 78-86.
    12. Gerrath, Maximilian H.E.E. & Mafael, Alexander & Ulqinaku, Aulona & Biraglia, Alessandro, 2023. "Service failures in times of crisis: An analysis of eWOM emotionality," Journal of Business Research, Elsevier, vol. 154(C).
    13. Akbaritabar, Aliakbar & Stephen, Dimity & Squazzoni, Flaminio, 2022. "A study of referencing changes in preprint-publication pairs across multiple fields," Journal of Informetrics, Elsevier, vol. 16(2).
    14. Alantari, Huwail J. & Currim, Imran S. & Deng, Yiting & Singh, Sameer, 2022. "An empirical comparison of machine learning methods for text-based sentiment analysis of online consumer reviews," International Journal of Research in Marketing, Elsevier, vol. 39(1), pages 1-19.
    15. Andrijana Perković Paloš & Antonija Mijatović & Ivan Buljan & Daniel Garcia-Costa & Elena Álvarez-García & Francisco Grimaldo & Ana Marušić, 2023. "Linguistic and semantic characteristics of articles and peer review reports in Social Sciences and Medical and Health Sciences: analysis of articles published in Open Research Central," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(8), pages 4707-4729, August.
    16. Andre Bruns & Niels Taubert, 2021. "Investigating the Blind Spot of a Monitoring System for Article Processing Charges," Publications, MDPI, vol. 9(3), pages 1-9, September.
    17. Jinhyo Joseph Yun & Zheng Liu & Euiseob Jeong & Sangwoo Kim & Kyunghun Kim, 2022. "The Difference in Open Innovation between Open Access and Closed Access, According to the Change of Collective Intelligence and Knowledge Amount," Sustainability, MDPI, vol. 14(5), pages 1-19, February.
    18. Minxian Zheng & Kuangji Zhao & Shikui Zhao & Yantong Zhang, 2020. "Effecting variables of journal’s ranking in forestry field," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 135-151, October.
    19. Roberta Ruggieri & Fabrizio Pecoraro & Daniela Luzi, 2021. "An intersectional approach to analyse gender productivity and open access: a bibliometric analysis of the Italian National Research Council," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1647-1673, February.
    20. Xie, Yundong & Wu, Qiang & Wang, Yezhu & Hou, Li & Liu, Yuanyuan, 2024. "Does the handling time of scientific papers relate to their academic impact and social attention? Evidence from Nature, Science, and PNAS," Journal of Informetrics, Elsevier, vol. 18(2).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:126:y:2021:i:7:d:10.1007_s11192-021-04032-8. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.