IDEAS home Printed from https://ideas.repec.org/a/tsj/stataj/v12y2012i4p605-622.html
   My bibliography  Save this article

Graphical augmentations to the funnel plot to assess the impact of a new study on an existing meta-analysis

Author

Listed:
  • Michael J. Crowther

    (University of Leicester)

  • Dean Langan

    (University of Leeds)

  • Alex J. Sutton

    (University of Leicester)

Abstract

Funnel plots are currently advocated to investigate the presence of publication bias (and other possible sources of bias) in meta-analysis. A previously described augmentation to the funnel plot—to aid its interpretation in assessing publication biases—is the addition of statistical contours indicating regions where studies would have to be for a given level of significance, as implemented in the Stata package confunnel by Palmer et al. (2008, Stata Journal 8: 242–254). In this article, we describe http://www.stata-journal.com/software/the implementation of a new range of overlay aug- mentations to the funnel plot, many described in detail recently by Langan et al. (2012, Journal of Clinical Epidemiology 65: 511–519). The purpose of these over- lays is to display the potential impact a new study would have on an existing meta-analysis, providing an indication of the robustness of the meta-analysis to the addition of new evidence. Thus these overlays extend the use of the funnel plot beyond assessments of publication biases. Two main graphical displays are described: 1) statistical significance contours, which define regions of the funnel plot where a new study would have to be located to change the statistical signifi- cance of the meta-analysis; and 2) heterogeneity contours, which show how a new study would affect the extent of heterogeneity in a given meta-analysis. We present the extfunnel command, which implements the methods of Langan et al. (2012, Journal of Clinical Epidemiology 65: 511–519), and, furthermore, we extend the graphical displays to illustrate the impact a new study has on lower and upper confidence interval values and the confidence interval width of the pooled meta-analytic result. We also describe http://www.stata-journal.com/software/overlays for the impact of a future study on user-defined limits of clinical equivalence. We implement inverse- variance weighted methods by using both explicit formulas for contour lines and a simulation approach optimized in Mata. Copyright 2012 by StataCorp LP.

Suggested Citation

  • Michael J. Crowther & Dean Langan & Alex J. Sutton, 2012. "Graphical augmentations to the funnel plot to assess the impact of a new study on an existing meta-analysis," Stata Journal, StataCorp LP, vol. 12(4), pages 605-622, December.
    • Handle: RePEc:tsj:stataj:v:12:y:2012:i:4:p:605-622
    Note: to access software from within Stata, net describe http://www.stata-journal.com/software/sj12-4/gr0054/
    as

    Download full text from publisher

    File URL: http://www.stata-journal.com/article.html?article=gr0054
    File Function: link to article download
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Jonathan A.C. Sterne & Roger M. Harbord, 2004. "Funnel plots in meta-analysis," Stata Journal, StataCorp LP, vol. 4(2), pages 127-141, June.
    2. Tom M. Palmer & Jaime L. Peters & Alex J. Sutton & Santiago G. Moreno, 2008. "Contour-enhanced funnel plots for meta-analysis," Stata Journal, StataCorp LP, vol. 8(2), pages 242-254, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Nikolaos Terzidis & Steven Brakman & Raquel Ortega-Argiles, 2019. "Labour Markets, Trade and Technological Progress. A Meta-Study," CESifo Working Paper Series 7719, CESifo.
    2. Xue-bin Wang & Ning-hua Cui & Jie Yang & Xue-ping Qiu & Jia-jia Gao & Na Yang & Fang Zheng, 2014. "Angiotensin-Converting Enzyme Insertion/Deletion Polymorphism Is Not a Major Determining Factor in the Development of Sporadic Alzheimer Disease: Evidence from an Updated Meta-Analysis," PLOS ONE, Public Library of Science, vol. 9(10), pages 1-13, October.
    3. Pérez-Calero, Leticia & Hurtado-González, José Manuel & López-Iturriaga, Félix J., 2019. "Do the institutional environment and types of owners influence the relationship between ownership concentration and board of director independence? An international meta-analysis," International Review of Financial Analysis, Elsevier, vol. 61(C), pages 233-244.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Roger M. Harbord & Ross J. Harris & Jonathan A. C. Sterne, 2009. "Updated tests for small-study effects in meta-analyses," Stata Journal, StataCorp LP, vol. 9(2), pages 197-210, June.
    2. Wamisho, Kassu, 2013. "A Meta Regression Analysis of Soil Organic Carbon Sequestration in Corn Belt States: Implication for Cellulosic Biofuel Production," 2013 Annual Meeting, August 4-6, 2013, Washington, D.C. 149741, Agricultural and Applied Economics Association.
    3. Pérez-Calero, Leticia & Hurtado-González, José Manuel & López-Iturriaga, Félix J., 2019. "Do the institutional environment and types of owners influence the relationship between ownership concentration and board of director independence? An international meta-analysis," International Review of Financial Analysis, Elsevier, vol. 61(C), pages 233-244.
    4. Gundula Krack, 2019. "How to make value-based health insurance designs more effective? A systematic review and meta-analysis," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 20(6), pages 841-856, August.
    5. Amal F. Alshammary & Khalid Khalaf Alharbi & Naif Jameel Alshehri & Vishal Vennu & Imran Ali Khan, 2021. "Metabolic Syndrome and Coronary Artery Disease Risk: A Meta-Analysis of Observational Studies," IJERPH, MDPI, vol. 18(4), pages 1-16, February.
    6. Ugur, Mehmet & Trushin, Eshref & Solomon, Edna & Guidi, Francesco, 2016. "R&D and productivity in OECD firms and industries: A hierarchical meta-regression analysis," Research Policy, Elsevier, vol. 45(10), pages 2069-2086.
    7. Ruxiu Tie & Tiansong Zhang & Huarui Fu & Limengmeng Wang & Yebo Wang & Ying He & Binsheng Wang & Ni Zhu & Shan Fu & Xiaoyu Lai & Jimin Shi & He Huang, 2014. "Association between DNMT3A Mutations and Prognosis of Adults with De Novo Acute Myeloid Leukemia: A Systematic Review and Meta-Analysis," PLOS ONE, Public Library of Science, vol. 9(6), pages 1-11, June.
    8. Christin Höge-Junge & Stefan Eckert, 2024. "Multinationality and systematic risk: a literature review and meta-analysis," Management Review Quarterly, Springer, vol. 74(1), pages 377-414, February.
    9. Nino Fonseca & Marcelino Sánchez-Rivero, 2020. "Significance bias in the tourism-led growth literature," Tourism Economics, , vol. 26(1), pages 137-154, February.
    10. Ugur, Mehmet & Mitra, Arup, 2017. "Technology Adoption and Employment in Less Developed Countries: A Mixed-Method Systematic Review," World Development, Elsevier, vol. 96(C), pages 1-18.
    11. Carina Neisser, 2021. "The Elasticity of Taxable Income: A Meta-Regression Analysis [The top 1% in international and historical perspective]," The Economic Journal, Royal Economic Society, vol. 131(640), pages 3365-3391.
    12. Chun-Yu Chang & Po-Chen Lin & Yung-Jiun Chien & Chien-Sheng Chen & Meng-Yu Wu, 2020. "Analysis of Chest-Compression Depth and Full Recoil in Two Infant Chest-Compression Techniques Performed by a Single Rescuer: Systematic Review and Meta-Analysis," IJERPH, MDPI, vol. 17(11), pages 1-17, June.
    13. Haris S Vasiliadis & Adriani Nikolakopoulou & Ian Shrier & Michael P Lunn & Ruth Brassington & Rob J P Scholten & Georgia Salanti, 2015. "Endoscopic and Open Release Similarly Safe for the Treatment of Carpal Tunnel Syndrome. A Systematic Review and Meta-Analysis," PLOS ONE, Public Library of Science, vol. 10(12), pages 1-16, December.
    14. Alex R. Piquero & David P. Farrington & Brandon C. Welsh & Richard Tremblay & Wesley G. Jennings, 2008. "Effects of Early Family/Parent Training Programs on Antisocial Behavior & Delinquency," Campbell Systematic Reviews, John Wiley & Sons, vol. 4(1), pages 1-122.
    15. Asim,Salman & Chase,Robert S. & Dar,Amit & Schmillen,Achim Daniel, 2015. "Improving education outcomes in South Asia : findings from a decade of impact evaluations," Policy Research Working Paper Series 7362, The World Bank.
    16. Jamal Hallajzadeh & Saeid Safiri & Mohammad Ali Mansournia & Maliheh Khoramdad & Neda Izadi & Amir Almasi-Hashiani & Reza Pakzad & Erfan Ayubi & Mark J M Sullman & Nahid Karamzad, 2017. "Metabolic syndrome and its components among rheumatoid arthritis patients: A comprehensive updated systematic review and meta-analysis," PLOS ONE, Public Library of Science, vol. 12(3), pages 1-21, March.
    17. Smith, William C. & Anderson, Emily & Salinas, Daniel & Horvatek, Renata & Baker, David P., 2015. "A meta-analysis of education effects on chronic disease: The causal dynamics of the Population Education Transition Curve," Social Science & Medicine, Elsevier, vol. 127(C), pages 29-40.
    18. Ivan Ropovik & Matus Adamkovic & David Greger, 2021. "Neglect of publication bias compromises meta-analyses of educational research," PLOS ONE, Public Library of Science, vol. 16(6), pages 1-14, June.
    19. Yun Hyung Koog & Seo Ryang We & Byung-Il Min, 2011. "Three-Armed Trials Including Placebo and No-Treatment Groups May Be Subject to Publication Bias: Systematic Review," PLOS ONE, Public Library of Science, vol. 6(5), pages 1-9, May.
    20. Simon Alcouffe & Jonathan Maurice & Nadine Galy & Loïc Gate, 2019. "Is the limited diffusion of management accounting innovations really a paradox? A meta-analysis of the relationship between product diversity and the adoption of Activity-Based Costing [La faible d," Post-Print hal-02124677, HAL.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:tsj:stataj:v:12:y:2012:i:4:p:605-622. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Christopher F. Baum or Lisa Gilmore (email available below). General contact details of provider: http://www.stata-journal.com/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.