IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0165889.html
   My bibliography  Save this article

The Impact of Excluding Trials from Network Meta-Analyses – An Empirical Study

Author

Listed:
  • Jing Zhang
  • Yiping Yuan
  • Haitao Chu

Abstract

Network meta-analysis (NMA) expands the scope of a conventional pairwise meta-analysis to simultaneously compare multiple treatments, which has an inherent appeal for clinicians, patients, and policy decision makers. Two recent reports have shown that the impact of excluding a treatment on NMAs can be substantial. However, no one has assessed the impact of excluding a trial from NMAs, which is important because many NMAs selectively include trials in the analysis. This article empirically examines the impact of trial exclusion using both the arm-based (AB) and contrast-based (CB) approaches, by reanalyzing 20 published NMAs involving 725 randomized controlled trials and 449,325 patients. For the population-averaged absolute risk estimates using the AB approach, the average fold changes across all networks ranged from 1.004 (with standard deviation 0.004) to 1.072 (with standard deviation 0.184); while the maximal fold changes ranged from 1.032 to 2.349. In 12 out of 20 NMAs, a 1.20-fold or larger change is observed in at least one of the population-averaged absolute risk estimates. In addition, while excluding a trial can substantially change the estimated relative effects (e.g., log odds ratios), there is no systematic difference in terms of changes between the two approaches. Changes in treatment rankings are observed in 7 networks and changes in inconsistency are observed in 3 networks. We do not observe correlations between changes in treatment effects, treatment rankings and inconsistency. Finally, we recommend rigorous inclusion and exclusion criteria, logical study selection process, and reasonable network geometry to ensure robustness and generalizability of the results of NMAs.

Suggested Citation

  • Jing Zhang & Yiping Yuan & Haitao Chu, 2016. "The Impact of Excluding Trials from Network Meta-Analyses – An Empirical Study," PLOS ONE, Public Library of Science, vol. 11(12), pages 1-17, December.
    • Handle: RePEc:plo:pone00:0165889
    DOI: 10.1371/journal.pone.0165889
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0165889
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0165889&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0165889?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Lu, Guobing & Ades, A.E., 2006. "Assessing Evidence Inconsistency in Mixed Treatment Comparisons," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 447-459, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. H. P. Piepho & E. R. Williams & L. V. Madden, 2012. "The Use of Two-Way Linear Mixed Models in Multitreatment Meta-Analysis," Biometrics, The International Biometric Society, vol. 68(4), pages 1269-1277, December.
    2. A. Goubar & A. E. Ades & D. De Angelis & C. A. McGarrigle & C. H. Mercer & P. A. Tookey & K. Fenton & O. N. Gill, 2008. "Estimates of human immunodeficiency virus prevalence and proportion diagnosed based on Bayesian multiparameter synthesis of surveillance data," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 171(3), pages 541-580, June.
    3. David M. Phillippo & Sofia Dias & A. E. Ades & Mark Belger & Alan Brnabic & Alexander Schacht & Daniel Saure & Zbigniew Kadziola & Nicky J. Welton, 2020. "Multilevel network meta‐regression for population‐adjusted treatment comparisons," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 183(3), pages 1189-1210, June.
    4. Howard Thom & Frank Ender & Saisudha Samavedam & Caridad Perez Vivez & Subhajit Gupta & Mukesh Dhariwal & Jan de Haan & Derek O’Boyle, 2019. "Effect of AcrySof versus other intraocular lens properties on the risk of Nd:YAG capsulotomy after cataract surgery: A systematic literature review and network meta-analysis," PLOS ONE, Public Library of Science, vol. 14(8), pages 1-15, August.
    5. S. Dias & N. J. Welton & V. C. C. Marinho & G. Salanti & J. P. T. Higgins & A. E. Ades, 2010. "Estimation and adjustment of bias in randomized evidence by using mixed treatment comparison meta‐analysis," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 173(3), pages 613-629, July.
    6. Dan Jackson & Sylwia Bujkiewicz & Martin Law & Richard D. Riley & Ian R. White, 2018. "A matrix†based method of moments for fitting multivariate network meta†analysis models with multiple outcomes and random inconsistency effects," Biometrics, The International Biometric Society, vol. 74(2), pages 548-556, June.
    7. Hwanhee Hong & Bradley P. Carlin & Tatyana A. Shamliyan & Jean F. Wyman & Rema Ramakrishnan & François Sainfort & Robert L. Kane, 2013. "Comparing Bayesian and Frequentist Approaches for Multiple Outcome Mixed Treatment Comparisons," Medical Decision Making, , vol. 33(5), pages 702-714, July.
    8. Giammarco Alderotti & Daniele Vignoli & Michela Baccini & Anna Matysiak, 2019. "Employment Uncertainty and Fertility: A Network Meta-Analysis of European Research Findings," Econometrics Working Papers Archive 2019_06, Universita' degli Studi di Firenze, Dipartimento di Statistica, Informatica, Applicazioni "G. Parenti".
    9. Pepijn Vemer & Maiwenn J Al & Mark Oppe & Maureen P M H Rutten-van Mölken, 2017. "Mix and match. A simulation study on the impact of mixed-treatment comparison methods on health-economic outcomes," PLOS ONE, Public Library of Science, vol. 12(2), pages 1-20, February.
    10. Peixia Cheng & Liheng Tan & Peishan Ning & Li Li & Yuyan Gao & Yue Wu & David C. Schwebel & Haitao Chu & Huaiqiong Yin & Guoqing Hu, 2018. "Comparative Effectiveness of Published Interventions for Elderly Fall Prevention: A Systematic Review and Network Meta-Analysis," IJERPH, MDPI, vol. 15(3), pages 1-14, March.
    11. Tailai He & Fei Han & Jiahao Wang & Yihe Hu & Jianxi Zhu, 2021. "Efficacy and safety of anticoagulants for postoperative thrombophylaxis in total hip and knee arthroplasty: A PRISMA-compliant Bayesian network meta-analysis," PLOS ONE, Public Library of Science, vol. 16(6), pages 1-14, June.
    12. Anna Chaimani & Julian P T Higgins & Dimitris Mavridis & Panagiota Spyridonos & Georgia Salanti, 2013. "Graphical Tools for Network Meta-Analysis in STATA," PLOS ONE, Public Library of Science, vol. 8(10), pages 1-12, October.
    13. Gretchen Bjornstad & Shreya Sonthalia & Benjamin Rouse & Leanne Freeman & Natasha Hessami & Jo Hickman Dunne & Nick Axford, 2024. "A comparison of the effectiveness of cognitive behavioural interventions based on delivery features for elevated symptoms of depression in adolescents: A systematic review," Campbell Systematic Reviews, John Wiley & Sons, vol. 20(1), March.
    14. J.Jaime Caro & K. Ishak, 2010. "No Head-to-Head Trial? Simulate the Missing Arms," PharmacoEconomics, Springer, vol. 28(10), pages 957-967, October.
    15. Gretchen J. Bjornstad & Shreya Sonthalia & Benjamin Rouse & Luke Timmons & Laura Whybra & Nick Axford, 2020. "PROTOCOL: A comparison of the effectiveness of cognitive behavioural interventions based on delivery features for elevated symptoms of depression in adolescents," Campbell Systematic Reviews, John Wiley & Sons, vol. 16(1), March.
    16. van Valkenhoef, Gert & de Brock, E.O. & Hillege, Hans & Zhao, Jing, 2012. "Addis," Research Report 12007-Other, University of Groningen, Research Institute SOM (Systems, Organisations and Management).
    17. Loukia M. Spineli, 2022. "A Revised Framework to Evaluate the Consistency Assumption Globally in a Network of Interventions," Medical Decision Making, , vol. 42(5), pages 637-648, July.
    18. Lin, Lifeng & Zhang, Jing & Hodges, James S. & Chu, Haitao, 2017. "Performing Arm-Based Network Meta-Analysis in R with the pcnetmeta Package," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 80(i05).
    19. David Lunn & Jessica Barrett & Michael Sweeting & Simon Thompson, 2013. "Fully Bayesian hierarchical modelling in two stages, with application to meta-analysis," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 62(4), pages 551-572, August.
    20. repec:dgr:rugsom:12007-0ther is not listed on IDEAS
    21. David M. Phillippo & Sofia Dias & A. E. Ades & Vanessa Didelez & Nicky J. Welton, 2018. "Sensitivity of treatment recommendations to bias in network meta‐analysis," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 181(3), pages 843-867, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0165889. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.