IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v95y2013i3d10.1007_s11192-012-0845-z.html
   My bibliography  Save this article

Citation gamesmanship: testing for evidence of ego bias in peer review

Author

Listed:
  • Cassidy R. Sugimoto

    (Indiana University)

  • Blaise Cronin

    (Indiana University)

Abstract

This study tests for evidence of gaming and attention mongering—here termed ego bias—in the scholarly peer review process. We explore the extent to which authors cite the target journal and its editor and also the relationship between targeted references and editorial decisions. We examine referee reports for the presence and type of references and determine the extent to which reviewers cite their own work in their reports. Our results are based on a sample of 442 manuscripts and 927 referee reports submitted to the Journal of the American Society of Information Science and Technology. We find little evidence that editors, authors or reviewers use the peer review process as an opportunity to play citation games.

Suggested Citation

  • Cassidy R. Sugimoto & Blaise Cronin, 2013. "Citation gamesmanship: testing for evidence of ego bias in peer review," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(3), pages 851-862, June.
    • Handle: RePEc:spr:scient:v:95:y:2013:i:3:d:10.1007_s11192-012-0845-z
    DOI: 10.1007/s11192-012-0845-z
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-012-0845-z
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-012-0845-z?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Blaise Cronin, 2012. "Do me a favor," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(7), pages 1281-1281, July.
    2. Lutz Bornmann & Hans-Dieter Daniel, 2009. "Reviewer and editor biases in journal peer review: an investigation of manuscript refereeing at Angewandte Chemie International Edition," Research Evaluation, Oxford University Press, vol. 18(4), pages 262-272, October.
    3. Tove Faber Frandsen & Jeppe Nicolaisen, 2011. "Praise the bridge that carries you over: Testing the flattery citation hypothesis," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 62(5), pages 807-818, May.
    4. Upali W. Jayasinghe & Herbert W. Marsh & Nigel Bond, 2003. "A multilevel cross‐classified modelling approach to peer review of grant proposals: the effects of assessor and researcher attributes on assessor ratings," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 166(3), pages 279-300, October.
    5. Lutz Bornmann & Markus Wolf & Hans-Dieter Daniel, 2012. "Closed versus open reviewing of journal manuscripts: how far do comments differ in language use?," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(3), pages 843-856, June.
    6. Michael H. MacRoberts & Barbara R. MacRoberts, 1989. "Problems of citation analysis: A critical review," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 40(5), pages 342-349, September.
    7. Ángel Borrego & Maite Barrios & Anna Villarroya & Candela Ollé, 2010. "Scientific output and impact of postdoctoral scientists: a gender perspective," Scientometrics, Springer;Akadémiai Kiadó, vol. 83(1), pages 93-101, April.
    8. Juan Miguel Campanario & Erika Acedo, 2007. "Rejecting highly cited papers: The views of scientists who encounter resistance to their discoveries from other scientists," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 58(5), pages 734-743, March.
    9. Lutz Bornmann & Christophe Weymuth & Hans-Dieter Daniel, 2010. "A content analysis of referees’ comments: how do comments on manuscripts rejected by a high-impact journal and later published in either a low- or high-impact journal differ?," Scientometrics, Springer;Akadémiai Kiadó, vol. 83(2), pages 493-506, May.
    10. Matthew RE Symonds & Neil J Gemmell & Tamsin L Braisher & Kylie L Gorringe & Mark A Elgar, 2006. "Gender Differences in Publication Output: Towards an Unbiased Metric of Research Performance," PLOS ONE, Public Library of Science, vol. 1(1), pages 1-5, December.
    11. Juan Miguel Campanario, 1996. "The competition for journal space among referees, editors, and other authors and its influence on journals' impact factors," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 47(3), pages 184-192, March.
    12. Jerome K. Vanclay, 2009. "Bias in the journal impact factor," Scientometrics, Springer;Akadémiai Kiadó, vol. 78(1), pages 3-12, January.
    13. Blaise Cronin, 2012. "Do me a favor," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 63(7), pages 1281-1281, July.
    14. Laband, David N & Piette, Michael J, 1994. "Favoritism versus Search for Good Papers: Empirical Evidence Regarding the Behavior of Journal Editors," Journal of Political Economy, University of Chicago Press, vol. 102(1), pages 194-203, February.
    15. Tove Faber Frandsen & Jeppe Nicolaisen, 2011. "Praise the bridge that carries you over: Testing the flattery citation hypothesis," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 62(5), pages 807-818, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Sugimoto, Cassidy R. & Larivière, Vincent & Ni, Chaoqun & Cronin, Blaise, 2013. "Journal acceptance rates: A cross-disciplinary analysis of variability and relationships with journal measures," Journal of Informetrics, Elsevier, vol. 7(4), pages 897-906.
    2. Stelmakh, Ivan & Rastogi, Charvi & Liu, Ryan & Chawla, Shuchi & Shah, Nihar & Echenique, Federico, 2023. "Cite-seeing and reviewing: A study on citation bias in peer review," Department of Economics, Working Paper Series qt3883h8j1, Department of Economics, Institute for Business and Economic Research, UC Berkeley.
    3. J. A. García & Rosa Rodriguez-Sánchez & J. Fdez-Valdivia, 2016. "Why the referees’ reports I receive as an editor are so much better than the reports I receive as an author?," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(3), pages 967-986, March.
    4. Eduardo Kunzel Teixeira & Mirian Oliveira, 2018. "Editorial board interlocking in knowledge management and intellectual capital research field," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(3), pages 1853-1869, December.
    5. Shuo Xu & Mengjia An & Xin An, 2021. "Do scientific publications by editorial board members have shorter publication delays and then higher influence?," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(8), pages 6697-6713, August.
    6. Wang, Qi & Sandström, Ulf, 2014. "Defining the Role of Cognitive Distance in the Peer Review Process: Explorative Study of a Grant Scheme in Infection Biology," INDEK Working Paper Series 2014/10, Royal Institute of Technology, Department of Industrial Economics and Management.
    7. Sergio Copiello, 2018. "On the money value of peer review," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 613-620, April.
    8. Kevin J. Boudreau & Eva C. Guinan & Karim R. Lakhani & Christoph Riedl, 2016. "Looking Across and Looking Beyond the Knowledge Frontier: Intellectual Distance, Novelty, and Resource Allocation in Science," Management Science, INFORMS, vol. 62(10), pages 2765-2783, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Carole J. Lee & Cassidy R. Sugimoto & Guo Zhang & Blaise Cronin, 2013. "Bias in peer review," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(1), pages 2-17, January.
    2. Sugimoto, Cassidy R. & Larivière, Vincent & Ni, Chaoqun & Cronin, Blaise, 2013. "Journal acceptance rates: A cross-disciplinary analysis of variability and relationships with journal measures," Journal of Informetrics, Elsevier, vol. 7(4), pages 897-906.
    3. Jacob B. Slyder & Beth R. Stein & Brent S. Sams & David M. Walker & B. Jacob Beale & Jeffrey J. Feldhaus & Carolyn A. Copenheaver, 2011. "Citation pattern and lifespan: a comparison of discipline, institution, and individual," Scientometrics, Springer;Akadémiai Kiadó, vol. 89(3), pages 955-966, December.
    4. Eduardo Kunzel Teixeira & Mirian Oliveira, 2018. "Editorial board interlocking in knowledge management and intellectual capital research field," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(3), pages 1853-1869, December.
    5. Allen Bellas & Lea-Rachel Kosnik, 2019. "Which leading journal leads? Idea diffusion in economics research journals," Empirical Economics, Springer, vol. 57(3), pages 901-921, September.
    6. Joshua Aizenman & Kenneth Kletzer, 2011. "The life cycle of scholars and papers in economics - the 'citation death tax'," Applied Economics, Taylor & Francis Journals, vol. 43(27), pages 4135-4148.
    7. K. C. Garg & S. Kumar, 2014. "Scientometric profile of Indian scientific output in life sciences with a focus on the contributions of women scientists," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(3), pages 1771-1783, March.
    8. Stephen A Gallo & Joanne H Sullivan & Scott R Glisson, 2016. "The Influence of Peer Reviewer Expertise on the Evaluation of Research Funding Applications," PLOS ONE, Public Library of Science, vol. 11(10), pages 1-18, October.
    9. Alberto Baccini & Lucio Barabesi, 2010. "Interlocking editorship. A network analysis of the links between economic journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(2), pages 365-389, February.
    10. Maite Barrios & Anna Villarroya & Ángel Borrego, 2013. "Scientific production in psychology: a gender analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(1), pages 15-23, April.
    11. Heidi Prozesky & Nelius Boshoff, 2012. "Bibliometrics as a tool for measuring gender-specific research performance: an example from South African invasion ecology," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(2), pages 383-406, February.
    12. Teplitskiy, Misha & Acuna, Daniel & Elamrani-Raoult, Aïda & Körding, Konrad & Evans, James, 2018. "The sociology of scientific validity: How professional networks shape judgement in peer review," Research Policy, Elsevier, vol. 47(9), pages 1825-1841.
    13. Tahereh Dehdarirad & Anna Villarroya & Maite Barrios, 2015. "Research on women in science and higher education: a bibliometric analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(3), pages 795-812, June.
    14. Marco Seeber & Jef Vlegels & Mattia Cattaneo, 2022. "Conditions that do or do not disadvantage interdisciplinary research proposals in project evaluation," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 73(8), pages 1106-1126, August.
    15. Kevin J. Boudreau & Eva C. Guinan & Karim R. Lakhani & Christoph Riedl, 2016. "Looking Across and Looking Beyond the Knowledge Frontier: Intellectual Distance, Novelty, and Resource Allocation in Science," Management Science, INFORMS, vol. 62(10), pages 2765-2783, October.
    16. Armstrong, J. Scott, 2003. "Discovery and communication of important marketing findings: Evidence and proposals," Journal of Business Research, Elsevier, vol. 56(1), pages 69-84, January.
    17. John Panaretos & Chrisovaladis Malesios, 2009. "Assessing scientific research performance and impact with single indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 81(3), pages 635-670, December.
    18. Shuo Xu & Mengjia An & Xin An, 2021. "Do scientific publications by editorial board members have shorter publication delays and then higher influence?," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(8), pages 6697-6713, August.
    19. Dongyu Zang & Chunli Liu, 2023. "Exploring the clinical translation intensity of papers published by the world’s top scientists in basic medicine," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(4), pages 2371-2416, April.
    20. Andreas Nishikawa-Pacher & Tamara Heck & Kerstin Schoch, 2023. "Open Editors: A dataset of scholarly journals’ editorial board positions," Research Evaluation, Oxford University Press, vol. 32(2), pages 228-243.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:95:y:2013:i:3:d:10.1007_s11192-012-0845-z. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.