IDEAS home Printed from https://ideas.repec.org/a/bla/jorssa/v173y2010i3p575-591.html
   My bibliography  Save this article

Assessing publication bias in meta‐analyses in the presence of between‐study heterogeneity

Author

Listed:
  • Jaime L. Peters
  • Alex J. Sutton
  • David R. Jones
  • Keith R. Abrams
  • Lesley Rushton
  • Santiago G. Moreno

Abstract

Summary. Between‐study heterogeneity and publication bias are common features of a meta‐analysis that can be present simultaneously. When both are suspected, consideration must be made of each in the assessment of the other. We consider extended funnel plot tests for detecting publication bias, and selection modelling and trim‐and‐fill methods to adjust for publication bias in the presence of between‐study heterogeneity. These methods are applied to two example data sets. Results indicate that ignoring between‐study heterogeneity when assessing publication bias can be misleading, but that methods to test or adjust for publication bias in the presence of heterogeneity may not be powerful when the meta‐analysis is not large. It is therefore unrealistic to expect to disentangle the effects of publication bias and heterogeneity reliably in all except the largest meta‐analyses.

Suggested Citation

  • Jaime L. Peters & Alex J. Sutton & David R. Jones & Keith R. Abrams & Lesley Rushton & Santiago G. Moreno, 2010. "Assessing publication bias in meta‐analyses in the presence of between‐study heterogeneity," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 173(3), pages 575-591, July.
    • Handle: RePEc:bla:jorssa:v:173:y:2010:i:3:p:575-591
    DOI: 10.1111/j.1467-985X.2009.00629.x
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/j.1467-985X.2009.00629.x
    Download Restriction: no
    File URL: https://libkey.io/10.1111/j.1467-985X.2009.00629.x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. J. Copas, 1999. "What works?: selectivity models and meta‐analysis," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 162(1), pages 95-109.
    2. Sue Duval & Richard Tweedie, 2000. "Trim and Fill: A Simple Funnel-Plot–Based Method of Testing and Adjusting for Publication Bias in Meta-Analysis," Biometrics, The International Biometric Society, vol. 56(2), pages 455-463, June.
    3. Larry V. Hedges & Jack L. Vevea, 1996. "Estimating Effect Size Under Publication Bias: Small Sample Properties and Robustness of a Random Effects Selection Model," Journal of Educational and Behavioral Statistics, , vol. 21(4), pages 299-332, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Polák, Petr, 2017. "The productivity paradox: A meta-analysis," Information Economics and Policy, Elsevier, vol. 38(C), pages 38-54.
    2. Sergio Nolazco & Kaspar Delhey & Shinichi Nakagawa & Anne Peters, 2022. "Ornaments are equally informative in male and female birds," Nature Communications, Nature, vol. 13(1), pages 1-10, December.
    3. Celbis, Mehmet Güney & Nijkamp, Peter & Poot, Jacques, 2013. "How big is the impact of infrastructure on trade? Evidence from meta-analysis," MERIT Working Papers 2013-032, United Nations University - Maastricht Economic and Social Research Institute on Innovation and Technology (MERIT).
    4. Dan Jackson, 2018. "Discussion on Quantifying publication bias in meta‐analysis," Biometrics, The International Biometric Society, vol. 74(3), pages 795-796, September.
    5. Augusteijn, Hilde & van Aert, Robbie Cornelis Maria & van Assen, Marcel A. L. M., 2017. "The Effect of Publication Bias on the Assessment of Heterogeneity," OSF Preprints gv25c, Center for Open Science.
    6. Anna Chaimani & Julian P T Higgins & Dimitris Mavridis & Panagiota Spyridonos & Georgia Salanti, 2013. "Graphical Tools for Network Meta-Analysis in STATA," PLOS ONE, Public Library of Science, vol. 8(10), pages 1-12, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ao Huang & Kosuke Morikawa & Tim Friede & Satoshi Hattori, 2023. "Adjusting for publication bias in meta‐analysis via inverse probability weighting using clinical trial registries," Biometrics, The International Biometric Society, vol. 79(3), pages 2089-2102, September.
    2. Sanghyun Hong & W. Robert Reed, 2020. "Using Monte Carlo Experiments to Select Meta-Analytic Estimators," Working Papers in Economics 20/10, University of Canterbury, Department of Economics and Finance.
    3. van Aert, Robbie Cornelis Maria & van Assen, Marcel A. L. M., 2018. "P-uniform," MetaArXiv zqjr9, Center for Open Science.
    4. Mengke Li & Yukun Liu & Pengfei Li & Jing Qin, 2022. "Empirical likelihood meta-analysis with publication bias correction under Copas-like selection model," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 74(1), pages 93-112, February.
    5. Robbie C M van Aert & Jelte M Wicherts & Marcel A L M van Assen, 2019. "Publication bias examined in meta-analyses from psychology and medicine: A meta-meta-analysis," PLOS ONE, Public Library of Science, vol. 14(4), pages 1-32, April.
    6. Irsova, Zuzana & Bom, Pedro Ricardo Duarte & Havranek, Tomas & Rachinger, Heiko, 2023. "Spurious Precision in Meta-Analysis," MetaArXiv 3qp2w, Center for Open Science.
    7. van Aert, Robbie Cornelis Maria, 2018. "Dissertation R.C.M. van Aert," MetaArXiv eqhjd, Center for Open Science.
    8. Qian Li & Yan Chen & Shikun Sun & Muyuan Zhu & Jing Xue & Zihan Gao & Jinfeng Zhao & Yihe Tang, 2022. "Research on Crop Irrigation Schedules Under Deficit Irrigation—A Meta-analysis," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 36(12), pages 4799-4817, September.
    9. Bart Verkuil & Serpil Atasayi & Marc L Molendijk, 2015. "Workplace Bullying and Mental Health: A Meta-Analysis on Cross-Sectional and Longitudinal Data," PLOS ONE, Public Library of Science, vol. 10(8), pages 1-16, August.
    10. Damiano Pizzol & Mike Trott & Igor Grabovac & Mario Antunes & Anna Claudia Colangelo & Simona Ippoliti & Cristian Petre Ilie & Anne Carrie & Nicola Veronese & Lee Smith, 2021. "Laparoscopy in Low-Income Countries: 10-Year Experience and Systematic Literature Review," IJERPH, MDPI, vol. 18(11), pages 1-11, May.
    11. Wolfgang Goymann & John C. Wingfield, 2014. "Male-to-female testosterone ratios, dimorphism, and life history—what does it really tell us?," Behavioral Ecology, International Society for Behavioral Ecology, vol. 25(4), pages 685-699.
    12. Alderotti, Giammarco & Rapallini, Chiara & Traverso, Silvio, 2023. "The Big Five personality traits and earnings: A meta-analysis," Journal of Economic Psychology, Elsevier, vol. 94(C).
    13. Ünal, Zehra E. & Kartal, Gamze & Ulusoy, Serra & Ala, Aslı M. & Yilmaz, Munube & Geary, David C., 2023. "Relative contributions of g and basic domain-specific mathematics skills to complex mathematics competencies," Intelligence, Elsevier, vol. 101(C).
    14. Daniele Zago & Maria Eugênia Andrighetto Canozzi & Júlio Otávio Jardim Barcellos, 2020. "Pregnant beef cow’s nutrition and its effects on postnatal weight and carcass quality of their progeny," PLOS ONE, Public Library of Science, vol. 15(8), pages 1-20, August.
    15. Viktoria Maria Baumeister & Leonie Petra Kuen & Maike Bruckes & Gerhard Schewe, 2021. "The Relationship of Work-Related ICT Use With Well-being, Incorporating the Role of Resources and Demands: A Meta-Analysis," SAGE Open, , vol. 11(4), pages 21582440211, November.
    16. Gundula Krack, 2019. "How to make value-based health insurance designs more effective? A systematic review and meta-analysis," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 20(6), pages 841-856, August.
    17. Angélica Pigola & Priscila Rezende Costa, 2022. "In search of understanding about knowledge and learning on innovation performance," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(7), pages 3995-4022, July.
    18. Christopher Snyder & Ran Zhuo, 2018. "Sniff Tests as a Screen in the Publication Process: Throwing out the Wheat with the Chaff," NBER Working Papers 25058, National Bureau of Economic Research, Inc.
    19. Chuang Yuan & Jing Wang & Michael Ying, 2016. "Predictive Value of Carotid Distensibility Coefficient for Cardiovascular Diseases and All-Cause Mortality: A Meta-Analysis," PLOS ONE, Public Library of Science, vol. 11(4), pages 1-15, April.
    20. Francisco Javier Blanco-Encomienda & Rocío García-Cantero & María José Latorre-Medina, 2020. "Association between Work-Related Rumination, Work Environment and Employee Well-Being: A Meta-Analytic Study of Main and Moderator Effects," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 150(3), pages 887-910, August.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssa:v:173:y:2010:i:3:p:575-591. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/rssssea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.