IDEAS home Printed from https://ideas.repec.org/a/spr/climat/v140y2017i3d10.1007_s10584-016-1880-1.html
   My bibliography  Save this article

No evidence of publication bias in climate change science

Author

Listed:
  • Christian Harlos

    (Lund University)

  • Tim C. Edgell

    (Stantec)

  • Johan Hollander

    (Lund University)

Abstract

Non-significant results are less likely to be reported by authors and, when submitted for peer review, are less likely to be published by journal editors. This phenomenon, known collectively as publication bias, is seen in a variety of scientific disciplines and can erode public trust in the scientific method and the validity of scientific theories. Public trust in science is especially important for fields like climate change science, where scientific consensus can influence state policies on a global scale, including strategies for industrial and agricultural management and development. Here, we used meta-analysis to test for biases in the statistical results of climate change articles, including 1154 experimental results from a sample of 120 articles. Funnel plots revealed no evidence of publication bias given no pattern of non-significant results being under-reported, even at low sample sizes. However, we discovered three other types of systematic bias relating to writing style, the relative prestige of journals, and the apparent rise in popularity of this field: First, the magnitude of statistical effects was significantly larger in the abstract than the main body of articles. Second, the difference in effect sizes in abstracts versus main body of articles was especially pronounced in journals with high impact factors. Finally, the number of published articles about climate change and the magnitude of effect sizes therein both increased within 2 years of the seminal report by the Intergovernmental Panel on Climate Change 2007.

Suggested Citation

  • Christian Harlos & Tim C. Edgell & Johan Hollander, 2017. "No evidence of publication bias in climate change science," Climatic Change, Springer, vol. 140(3), pages 375-385, February.
    • Handle: RePEc:spr:climat:v:140:y:2017:i:3:d:10.1007_s10584-016-1880-1
    DOI: 10.1007/s10584-016-1880-1
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10584-016-1880-1
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10584-016-1880-1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Daniele Fanelli, 2012. "Negative results are disappearing from most disciplines and countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(3), pages 891-904, March.
    2. David Holland, 2007. "Bias and Concealment in the IPCC Process: The “Hockey-Stick†Affair and its Implications," Energy & Environment, , vol. 18(7), pages 951-983, December.
    3. Michal Kicinski, 2013. "Publication Bias in Recent Meta-Analyses," PLOS ONE, Public Library of Science, vol. 8(11), pages 1-1, November.
    4. Daniele Fanelli, 2013. "Positive results receive more citations, but only in some disciplines," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(2), pages 701-709, February.
    5. Adam Corner & Lorraine Whitmarsh & Dimitrios Xenias, 2012. "Uncertainty, scepticism and attitudes towards climate change: biased assimilation and attitude polarisation," Climatic Change, Springer, vol. 114(3), pages 463-478, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Vitor Azevedo & Christopher Hoegner, 2023. "Enhancing stock market anomalies with machine learning," Review of Quantitative Finance and Accounting, Springer, vol. 60(1), pages 195-230, January.
    2. Augusteijn, Hilde & van Aert, Robbie Cornelis Maria & van Assen, Marcel A. L. M., 2017. "The Effect of Publication Bias on the Assessment of Heterogeneity," OSF Preprints gv25c, Center for Open Science.
    3. Karin Langenkamp & Bodo Rödel & Kerstin Taufenbach & Meike Weiland, 2018. "Open Access in Vocational Education and Training Research," Publications, MDPI, vol. 6(3), pages 1-12, July.
    4. Augusteijn, Hilde Elisabeth Maria & van Aert, Robbie Cornelis Maria & van Assen, Marcel A. L. M., 2021. "Posterior Probabilities of Effect Sizes and Heterogeneity in Meta-Analysis: An Intuitive Approach of Dealing with Publication Bias," OSF Preprints avkgj, Center for Open Science.
    5. Debra Javeline & Tracy Kijewski-Correa & Angela Chesler, 2019. "Does it matter if you “believe” in climate change? Not for coastal home vulnerability," Climatic Change, Springer, vol. 155(4), pages 511-532, August.
    6. Brian Fabo & Martina Jancokova & Elisabeth Kempf & Lubos Pastor, 2020. "Fifty Shades of QE: Conflicts of Interest in Economic Research," Working and Discussion Papers WP 5/2020, Research Department, National Bank of Slovakia.
    7. Douglas Guilbeault & Damon Centola, 2020. "Networked collective intelligence improves dissemination of scientific information regarding smoking risks," PLOS ONE, Public Library of Science, vol. 15(2), pages 1-14, February.
    8. Christian Heise & Joshua M. Pearce, 2020. "From Open Access to Open Science: The Path From Scientific Reality to Open Scientific Communication," SAGE Open, , vol. 10(2), pages 21582440209, May.
    9. Christine Wamsler & Jamie Bristow, 2022. "At the intersection of mind and climate change: integrating inner dimensions of climate change into policymaking and practice," Climatic Change, Springer, vol. 173(1), pages 1-22, July.
    10. András Szeberényi & Tomasz Rokicki & Árpád Papp-Váry, 2022. "Examining the Relationship between Renewable Energy and Environmental Awareness," Energies, MDPI, vol. 15(19), pages 1-25, September.
    11. Lorraine Whitmarsh & Dimitrios Xenias & Christopher R. Jones, 2019. "Framing effects on public support for carbon capture and storage," Palgrave Communications, Palgrave Macmillan, vol. 5(1), pages 1-10, December.
    12. Oliver Braganza, 2020. "A simple model suggesting economically rational sample-size choice drives irreproducibility," PLOS ONE, Public Library of Science, vol. 15(3), pages 1-19, March.
    13. Ganga Shreedhar & Susana Mourato, 2020. "Linking Human Destruction of Nature to COVID-19 Increases Support for Wildlife Conservation Policies," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 76(4), pages 963-999, August.
    14. Lawrence C. Hamilton, 2016. "Public Awareness of the Scientific Consensus on Climate," SAGE Open, , vol. 6(4), pages 21582440166, November.
    15. Bobtcheff, Catherine & Mariotti, Thomas & Levy, Raphaël, 2021. "Negative results in science: Blessing or (winner’s) curse," TSE Working Papers 21-1202, Toulouse School of Economics (TSE).
    16. Tobias Greitemeyer, 2014. "I Am Right, You Are Wrong: How Biased Assimilation Increases the Perceived Gap between Believers and Skeptics of Violent Video Game Effects," PLOS ONE, Public Library of Science, vol. 9(4), pages 1-7, April.
    17. Mei Tian & Yan Su & Xin Ru, 2016. "Perish or Publish in China: Pressures on Young Chinese Scholars to Publish in Internationally Indexed Journals," Publications, MDPI, vol. 4(2), pages 1-16, April.
    18. Lucia Savadori & Giuseppe Espa & Maria Michela Dickson, 2020. "The polarizing impact of numeracy, economic literacy, and science literacy on attitudes toward immigration," Papers 2011.02362, arXiv.org.
    19. Weziak-Bialowolska, Dorota & Lee, Matthew T. & Cowden, Richard G. & Bialowolski, Piotr & Chen, Ying & VanderWeele, Tyler J. & McNeely, Eileen, 2023. "Psychological caring climate at work, mental health, well-being, and work-related outcomes: Evidence from a longitudinal study and health insurance data," Social Science & Medicine, Elsevier, vol. 323(C).
    20. Daniele Fanelli & Rodrigo Costas & Vincent Larivière, 2015. "Misconduct Policies, Academic Culture and Career Stage, Not Gender or Pressures to Publish, Affect Scientific Integrity," PLOS ONE, Public Library of Science, vol. 10(6), pages 1-18, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:climat:v:140:y:2017:i:3:d:10.1007_s10584-016-1880-1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.