IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v94y2013i2d10.1007_s11192-012-0757-y.html
   My bibliography  Save this article

Positive results receive more citations, but only in some disciplines

Author

Listed:
  • Daniele Fanelli

    (The University of Edinburgh)

Abstract

Negative results are commonly assumed to attract fewer readers and citations, which would explain why journals in most disciplines tend to publish too many positive and statistically significant findings. This study verified this assumption by counting the citation frequencies of papers that, having declared to “test” a hypothesis, reported a “positive” (full or partial) or a “negative” (null or negative) support. Controlling for various confounders, positive results were cited on average 32 % more often. The citation advantage, however, was unequally distributed across disciplines (classified as in the Essential Science Indicators database). Using Space Science as the reference category, the citation differential was positive and formally statistically significant only in Neuroscience & Behaviour, Molecular Biology & Genetics, Clinical Medicine, and Plant and Animal Science. Overall, the effect was significantly higher amongst applied disciplines, and in the biological compared to the physical and the social sciences. The citation differential was not a significant predictor of the actual frequency of positive results amongst the 20 broad disciplines considered. Although future studies should attempt more fine-grained assessments, these results suggest that publication bias may have different causes and require different solutions depending on the field considered.

Suggested Citation

  • Daniele Fanelli, 2013. "Positive results receive more citations, but only in some disciplines," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(2), pages 701-709, February.
    • Handle: RePEc:spr:scient:v:94:y:2013:i:2:d:10.1007_s11192-012-0757-y
    DOI: 10.1007/s11192-012-0757-y
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-012-0757-y
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-012-0757-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Glenn D. Walters, 2006. "Predicting subsequent citations to articles published in twelve crime-psychology journals: Author impact versus journal impact," Scientometrics, Springer;Akadémiai Kiadó, vol. 69(3), pages 499-510, December.
    2. André Andrian Padial & João Carlos Nabout & Tadeu Siqueira & Luis Mauricio Bini & José Alexandre Felizola Diniz-Filho, 2010. "Weak evidence for determinants of citation frequency in ecological articles," Scientometrics, Springer;Akadémiai Kiadó, vol. 85(1), pages 1-12, October.
    3. Jonathan Knight, 2003. "Null and void," Nature, Nature, vol. 422(6932), pages 554-555, April.
    4. Marco Pautasso, 2010. "Worsening file-drawer problem in the abstracts of natural, medical and social science databases," Scientometrics, Springer;Akadémiai Kiadó, vol. 85(1), pages 193-202, October.
    5. Jonathan Schooler, 2011. "Unpublished results hide the decline effect," Nature, Nature, vol. 470(7335), pages 437-437, February.
    6. Chris (hristos) Doucouliagos & Patrice Laroche & T.D. Stanley, 2005. "Publication Bias in Union-Productivity Research? [¿ Publicación tendenciosa en la investigación sobre la productividad sindical ?]," Post-Print hal-02138187, HAL.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Christian Harlos & Tim C. Edgell & Johan Hollander, 2017. "No evidence of publication bias in climate change science," Climatic Change, Springer, vol. 140(3), pages 375-385, February.
    2. Ke, Qing, 2020. "The citation disadvantage of clinical research," Journal of Informetrics, Elsevier, vol. 14(1).
    3. Yifan Qian & Wenge Rong & Nan Jiang & Jie Tang & Zhang Xiong, 2017. "Citation regression analysis of computer science publications in different ranking categories and subfields," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(3), pages 1351-1374, March.
    4. Vitor Azevedo & Christopher Hoegner, 2023. "Enhancing stock market anomalies with machine learning," Review of Quantitative Finance and Accounting, Springer, vol. 60(1), pages 195-230, January.
    5. Lee Stapleton, 2015. "Do academics doubt their own research?," SPRU Working Paper Series 2015-24, SPRU - Science Policy Research Unit, University of Sussex Business School.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Daniele Fanelli, 2012. "Negative results are disappearing from most disciplines and countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(3), pages 891-904, March.
    2. John N. Parker & Stefano Allesina & Christopher J. Lortie, 2013. "Characterizing a scientific elite (B): publication and citation patterns of the most highly cited scientists in environmental science and ecology," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(2), pages 469-480, February.
    3. Mingyang Wang & Zhenyu Wang & Guangsheng Chen, 2019. "Which can better predict the future success of articles? Bibliometric indices or alternative metrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(3), pages 1575-1595, June.
    4. Martorell Cunil, Onofre & Otero González, Luis & Durán Santomil, Pablo & Mulet Forteza, Carlos, 2023. "How to accomplish a highly cited paper in the tourism, leisure and hospitality field," Journal of Business Research, Elsevier, vol. 157(C).
    5. Zhang, Mengya & Zhang, Gupeng & Liu, Yun & Zhai, Xiaorong & Han, Xinying, 2020. "Scientists’ genders and international academic collaboration: An empirical study of Chinese universities and research institutes," Journal of Informetrics, Elsevier, vol. 14(4).
    6. Iman Tahamtan & Askar Safipour Afshar & Khadijeh Ahamdzadeh, 2016. "Factors affecting number of citations: a comprehensive review of the literature," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(3), pages 1195-1225, June.
    7. Xueying Liu & Haoran Zhu, 2023. "Linguistic positivity in soft and hard disciplines: temporal dynamics, disciplinary variation, and the relationship with research impact," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(5), pages 3107-3127, May.
    8. Victor A Sanchez-Azanza & Raúl López-Penadés & Lucía Buil-Legaz & Eva Aguilar-Mediavilla & Daniel Adrover-Roig, 2017. "Is bilingualism losing its advantage? A bibliometric approach," PLOS ONE, Public Library of Science, vol. 12(4), pages 1-13, April.
    9. Ajiferuke, Isola & Famoye, Felix, 2015. "Modelling count response variables in informetric studies: Comparison among count, linear, and lognormal regression models," Journal of Informetrics, Elsevier, vol. 9(3), pages 499-513.
    10. Dimos, Christos & Pugh, Geoff & Hisarciklilar, Mehtap & Talam, Ema & Jackson, Ian, 2022. "The relative effectiveness of R&D tax credits and R&D subsidies: A comparative meta-regression analysis," Technovation, Elsevier, vol. 115(C).
    11. Demena, B.A., 2021. "Effectiveness of export promotion programmes," ISS Working Papers - General Series 688, International Institute of Social Studies of Erasmus University Rotterdam (ISS), The Hague.
    12. David I Stern, 2014. "High-Ranked Social Science Journal Articles Can Be Identified from Early Citation Information," PLOS ONE, Public Library of Science, vol. 9(11), pages 1-11, November.
    13. Nino Fonseca & Marcelino Sánchez-Rivero, 2020. "Significance bias in the tourism-led growth literature," Tourism Economics, , vol. 26(1), pages 137-154, February.
    14. Vieira, Valter Afonso & Rafael, Diego Nogueira & Agnihotri, Raj, 2022. "Augmented reality generalizations: A meta-analytical review on consumer-related outcomes and the mediating role of hedonic and utilitarian values," Journal of Business Research, Elsevier, vol. 151(C), pages 170-184.
    15. João Carlos Nabout & Micael Rosa Parreira & Fabrício Barreto Teresa & Fernanda Melo Carneiro & Hélida Ferreira Cunha & Luciana Souza Ondei & Samantha Salomão Caramori & Thannya Nascimento Soares, 2015. "Publish (in a group) or perish (alone): the trend from single- to multi-authorship in biological papers," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 357-364, January.
    16. Fanelli, Daniele, 2020. "Metascientific reproducibility patterns revealed by informatic measure of knowledge," MetaArXiv 5vnhj, Center for Open Science.
    17. Nicolas Vallois & Dorian Jullien, 2017. "Replication in experimental economics: A historical and quantitative approach focused on public good game experiments," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) halshs-01651080, HAL.
    18. Martin E Héroux & Janet L Taylor & Simon C Gandevia, 2015. "The Use and Abuse of Transcranial Magnetic Stimulation to Modulate Corticospinal Excitability in Humans," PLOS ONE, Public Library of Science, vol. 10(12), pages 1-10, December.
    19. Zehra Taşkın, 2021. "Forecasting the future of library and information science and its sub-fields," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1527-1551, February.
    20. Johannes Auer & Dominik Papies, 2020. "Cross-price elasticities and their determinants: a meta-analysis and new empirical generalizations," Journal of the Academy of Marketing Science, Springer, vol. 48(3), pages 584-605, May.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:94:y:2013:i:2:d:10.1007_s11192-012-0757-y. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.