IDEAS home Printed from https://ideas.repec.org/a/bla/jinfst/v73y2022i5p655-670.html
   My bibliography  Save this article

Classification and analysis of PubPeer comments: How a web journal club is used

Author

Listed:
  • José Luis Ortega

Abstract

This study explores the use of PubPeer by the scholarly community, to understand the issues discussed in an online journal club, the disciplines most commented on, and the characteristics of the most prolific users. A sample of 39,985 posts about 24,779 publications were extracted from PubPeer in 2019 and 2020. These comments were divided into seven categories according to their degree of seriousness (Positive review, Critical review, Lack of information, Honest errors, Methodological flaws, Publishing fraud, and Manipulation). The results show that more than two‐thirds of comments are posted to report some type of misconduct, mainly about image manipulation. These comments generate most discussion and take longer to be posted. By discipline, Health Sciences and Life Sciences are the most discussed research areas. The results also reveal “super commenters,” users who access the platform to systematically review publications. The study ends by discussing how various disciplines use the site for different purposes.

Suggested Citation

  • José Luis Ortega, 2022. "Classification and analysis of PubPeer comments: How a web journal club is used," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 73(5), pages 655-670, May.
    • Handle: RePEc:bla:jinfst:v:73:y:2022:i:5:p:655-670
    DOI: 10.1002/asi.24568
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/asi.24568
    Download Restriction: no
    File URL: https://libkey.io/10.1002/asi.24568?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Mike Thelwall & Kevan Buckley & Georgios Paltoglou, 2012. "Sentiment strength detection for the social web," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(1), pages 163-173, January.
    2. S. P. J. M. Horbach & W. Halffman, 2019. "The ability of different peer review procedures to flag problematic publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(1), pages 339-373, January.
    3. José Luis Ortega, 2017. "Are peer-review activities related to reviewer bibliometric performance? A scientometric analysis of Publons," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(2), pages 947-962, August.
    4. Ed Yong & Heidi Ledford & Richard Van Noorden, 2013. "Research ethics: 3 ways to blow the whistle," Nature, Nature, vol. 503(7477), pages 454-457, November.
    5. repec:nas:journl:v:115:y:2018:p:2952-2957 is not listed on IDEAS
    6. Charles W. Fox, 2017. "Difficulty of recruiting reviewers predicts review scores and editorial decisions at six journals of ecology and evolution," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 465-477, October.
    7. Mike Thelwall & Kevan Buckley & Georgios Paltoglou, 2012. "Sentiment strength detection for the social web," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 63(1), pages 163-173, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Abdelghani Maddi & Emmanuel Monneau & Catherine Guaspare-Cartron & Floriana Gargiulo & Michel Dubois, 2024. "Streetlight effect in PubPeer comments: are Open Access publications more scrutinized?," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(7), pages 4231-4247, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Xiong, Xi & Li, Yuanyuan & Qiao, Shaojie & Han, Nan & Wu, Yue & Peng, Jing & Li, Binyong, 2018. "An emotional contagion model for heterogeneous social media with multiple behaviors," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 490(C), pages 185-202.
    2. Kristina Lerman & Luciano G. Marin & Megha Arora & Lucas H. Costa Lima & Emilio Ferrara & David Garcia, 2018. "Language, demographics, emotions, and the structure of online social networks," Journal of Computational Social Science, Springer, vol. 1(1), pages 209-225, January.
    3. Ghasem Javadi & Mohammad Taleai, 2020. "Integration of User Generated Geo-contents and Official Data to Assess Quality of Life in Intra-national Level," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 152(1), pages 205-235, November.
    4. Sykora, Martin & Elayan, Suzanne & Hodgkinson, Ian R. & Jackson, Thomas W. & West, Andrew, 2022. "The power of emotions: Leveraging user generated content for customer experience management," Journal of Business Research, Elsevier, vol. 144(C), pages 997-1006.
    5. Agrawal, Shiv Ratan & Mittal, Divya, 2022. "Optimizing customer engagement content strategy in retail and E-tail: Available on online product review videos," Journal of Retailing and Consumer Services, Elsevier, vol. 67(C).
    6. Sunita Goel & Ozlem Uzuner, 2016. "Do Sentiments Matter in Fraud Detection? Estimating Semantic Orientation of Annual Reports," Intelligent Systems in Accounting, Finance and Management, John Wiley & Sons, Ltd., vol. 23(3), pages 215-239, July.
    7. Wouter van der Schors & Marco Varkevisser, 2023. "Does Enforcement of the Cartel Prohibition in Healthcare Reflect Public and Political Attitudes Towards Competition? A Longitudinal Study From the Netherlands," Journal of Competition Law and Economics, Oxford University Press, vol. 19(2), pages 193-219.
    8. Bandeh Ali Talpur & Declan O’Sullivan, 2020. "Cyberbullying severity detection: A machine learning approach," PLOS ONE, Public Library of Science, vol. 15(10), pages 1-19, October.
    9. Martin Haselmayer & Marcelo Jenny, 2017. "Sentiment analysis of political communication: combining a dictionary approach with crowdcoding," Quality & Quantity: International Journal of Methodology, Springer, vol. 51(6), pages 2623-2646, November.
    10. Balázs Győrffy & Andrea Magda Nagy & Péter Herman & Ádám Török, 2018. "Factors influencing the scientific performance of Momentum grant holders: an evaluation of the first 117 research groups," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(1), pages 409-426, October.
    11. Young Bin Kim & Sang Hyeok Lee & Shin Jin Kang & Myung Jin Choi & Jung Lee & Chang Hun Kim, 2015. "Virtual World Currency Value Fluctuation Prediction System Based on User Sentiment Analysis," PLOS ONE, Public Library of Science, vol. 10(8), pages 1-18, August.
    12. Booranakittipinyo, Amphai & Li, Rita Yi Man & Phakdeephirot, Nutteera, 2024. "Travelers' perception of smart airport facilities: An X (Twitter) sentiment analysis," Journal of Air Transport Management, Elsevier, vol. 118(C).
    13. Singh, Amit & Jenamani, Mamata & Thakkar, Jitesh J. & Rana, Nripendra P., 2022. "Quantifying the effect of eWOM embedded consumer perceptions on sales: An integrated aspect-level sentiment analysis and panel data modeling approach," Journal of Business Research, Elsevier, vol. 138(C), pages 52-64.
    14. Ping-Yu Hsu & Hong-Tsuen Lei & Shih-Hsiang Huang & Teng Hao Liao & Yao-Chung Lo & Chin-Chun Lo, 2019. "Effects of sentiment on recommendations in social network," Electronic Markets, Springer;IIM University of St. Gallen, vol. 29(2), pages 253-262, June.
    15. Fatma Najar & Nizar Bouguila, 2023. "On smoothing and scaling language model for sentiment based information retrieval," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 17(3), pages 725-744, September.
    16. Luis J. Callarisa-Fiol & Miguel Ángel Moliner-Tena & Rosa Rodríguez-Artola & Javier Sánchez-García, 2023. "Entrepreneurship innovation using social robots in tourism: a social listening study," Review of Managerial Science, Springer, vol. 17(8), pages 2945-2971, November.
    17. Gang Wang & Daqing Zheng & Shanlin Yang & Jian Ma, 2018. "FCE-SVM: a new cluster based ensemble method for opinion mining from social media," Information Systems and e-Business Management, Springer, vol. 16(4), pages 721-742, November.
    18. Qing Liu & Hosung Son, 2024. "Methods for aggregating investor sentiment from social media," Palgrave Communications, Palgrave Macmillan, vol. 11(1), pages 1-22, December.
    19. Mauricio Toledo-Acosta & Talin Barreiro & Asela Reig-Alamillo & Markus Müller & Fuensanta Aroca Bisquert & Maria Luisa Barrigon & Enrique Baca-Garcia & Jorge Hermosillo-Valadez, 2020. "Cognitive Emotional Embedded Representations of Text to Predict Suicidal Ideation and Psychiatric Symptoms," Mathematics, MDPI, vol. 8(11), pages 1-27, November.
    20. Phuong Nguyen Hong Huynh & Tin Trung Hoang & Huynh Thi Thuy Phan & Quynh Le Nhu Nguyen, 2024. "The customers’ perception of privacy in the retail industry when adopting digital transformation," HO CHI MINH CITY OPEN UNIVERSITY JOURNAL OF SCIENCE - ECONOMICS AND BUSINESS ADMINISTRATION, HO CHI MINH CITY OPEN UNIVERSITY JOURNAL OF SCIENCE, HO CHI MINH CITY OPEN UNIVERSITY, vol. 14(2), pages 143-159.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jinfst:v:73:y:2022:i:5:p:655-670. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.asis.org .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.