IDEAS home Printed from https://ideas.repec.org/a/bla/jorssa/v183y2020i1p193-209.html
   My bibliography  Save this article

Adjusting trial results for biases in meta‐analysis: combining data‐based evidence on bias with detailed trial assessment

Author

Listed:
  • K. M. Rhodes
  • J. Savović
  • R. Elbers
  • H. E. Jones
  • J. P. T. Higgins
  • J. A. C. Sterne
  • N. J. Welton
  • R. M. Turner

Abstract

Flaws in the conduct of randomized trials can lead to biased estimation of the intervention effect. Methods for adjustment of within‐trial biases in meta‐analysis include the use of empirical evidence from an external collection of meta‐analyses, and the use of expert opinion informed by the assessment of detailed trial information. Our aim is to present methods to combine these two approaches to gain the advantages of both. We make use of the risk of bias information that is routinely available in Cochrane reviews, by obtaining empirical distributions for the bias associated with particular bias profiles (combinations of risk of bias judgements). We propose three methods: a formal combination of empirical evidence and opinion in a Bayesian analysis; asking experts to give an opinion on bias informed by both summary trial information and a bias distribution from the empirical evidence, either numerically or by selecting areas of the empirical distribution. The methods are demonstrated through application to two example binary outcome meta‐analyses. Bias distributions based on opinion informed by trial information alone were most dispersed on average, and those based on opinions obtained by selecting areas of the empirical distribution were narrowest. Although the three methods for combining empirical evidence with opinion vary in ease and speed of implementation, they yielded similar results in the two examples.

Suggested Citation

  • K. M. Rhodes & J. Savović & R. Elbers & H. E. Jones & J. P. T. Higgins & J. A. C. Sterne & N. J. Welton & R. M. Turner, 2020. "Adjusting trial results for biases in meta‐analysis: combining data‐based evidence on bias with detailed trial assessment," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 183(1), pages 193-209, January.
    • Handle: RePEc:bla:jorssa:v:183:y:2020:i:1:p:193-209
    DOI: 10.1111/rssa.12485
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/rssa.12485
    Download Restriction: no
    File URL: https://libkey.io/10.1111/rssa.12485?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Sander Greenland, 2005. "Multiple‐bias modelling for analysis of observational data," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 168(2), pages 267-306, March.
    2. David M. Eddy, 1989. "The Confidence Profile Method: A Bayesian Method for Assessing Health Technologies," Operations Research, INFORMS, vol. 37(2), pages 210-228, April.
    3. N. J. Welton & A. E. Ades & J. B. Carlin & D. G. Altman & J. A. C. Sterne, 2009. "Models for potentially biased evidence in meta‐analysis using empirically based priors," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 172(1), pages 119-136, January.
    4. David M. Phillippo & Sofia Dias & A. E. Ades & Vanessa Didelez & Nicky J. Welton, 2018. "Sensitivity of treatment recommendations to bias in network meta‐analysis," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 181(3), pages 843-867, June.
    5. Rebecca M. Turner & David J. Spiegelhalter & Gordon C. S. Smith & Simon G. Thompson, 2009. "Bias modelling in evidence synthesis," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 172(1), pages 21-47, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. McCandless Lawrence C., 2012. "Meta-Analysis of Observational Studies with Unmeasured Confounders," The International Journal of Biostatistics, De Gruyter, vol. 8(2), pages 1-31, January.
    2. S. Dias & N. J. Welton & V. C. C. Marinho & G. Salanti & J. P. T. Higgins & A. E. Ades, 2010. "Estimation and adjustment of bias in randomized evidence by using mixed treatment comparison meta‐analysis," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 173(3), pages 613-629, July.
    3. Desiree C Wilks & Stephen J Sharp & Ulf Ekelund & Simon G Thompson & Adrian P Mander & Rebecca M Turner & Susan A Jebb & Anna Karin Lindroos, 2011. "Objectively Measured Physical Activity and Fat Mass in Children: A Bias-Adjusted Meta-Analysis of Prospective Studies," PLOS ONE, Public Library of Science, vol. 6(2), pages 1-8, February.
    4. Mathur, Maya B & VanderWeele, Tyler, 2018. "Statistical methods for evidence synthesis," Thesis Commons kd6ja, Center for Open Science.
    5. Ian Wadsworth & Lisa V. Hampson & Thomas Jaki & Graeme J. Sills & Anthony G. Marson & Richard Appleton, 2020. "A quantitative framework to inform extrapolation decisions in children," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 183(2), pages 515-534, February.
    6. Rebecca M Turner & Myfanwy Lloyd-Jones & Dilly O C Anumba & Gordon C S Smith & David J Spiegelhalter & Hazel Squires & John W Stevens & Michael J Sweeting & Stanislaw J Urbaniak & Robert Webster & Sim, 2012. "Routine Antenatal Anti-D Prophylaxis in Women Who Are Rh(D) Negative: Meta-Analyses Adjusted for Differences in Study Design and Quality," PLOS ONE, Public Library of Science, vol. 7(2), pages 1-10, February.
    7. Petra Schnell‐Inderst & Cynthia P. Iglesias & MARJAN Arvandi & ORIANA Ciani & Raffaella Matteucci Gothe & Jaime Peters & Ashley W. Blom & Rod S. Taylor & Uwe Siebert, 2017. "A bias‐adjusted evidence synthesis of RCT and observational data: the case of total hip replacement," Health Economics, John Wiley & Sons, Ltd., vol. 26(S1), pages 46-69, February.
    8. David M. Phillippo & Sofia Dias & A. E. Ades & Vanessa Didelez & Nicky J. Welton, 2018. "Sensitivity of treatment recommendations to bias in network meta‐analysis," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 181(3), pages 843-867, June.
    9. A. E. Ades & Karl Claxton & Mark Sculpher, 2006. "Evidence synthesis, parameter correlation and probabilistic sensitivity analysis," Health Economics, John Wiley & Sons, Ltd., vol. 15(4), pages 373-381, April.
    10. Lawrence C. McCandless & Sylvia Richardson & Nicky Best, 2012. "Adjustment for Missing Confounders Using External Validation Data and Propensity Scores," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 107(497), pages 40-51, March.
    11. Julian P. T. Higgins & Simon G. Thompson & David J. Spiegelhalter, 2009. "A re‐evaluation of random‐effects meta‐analysis," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 172(1), pages 137-159, January.
    12. Douglas E. Schaubel & Guanghui Wei, 2011. "Double Inverse-Weighted Estimation of Cumulative Treatment Effects Under Nonproportional Hazards and Dependent Censoring," Biometrics, The International Biometric Society, vol. 67(1), pages 29-38, March.
    13. Carlos Díaz-Venegas, 2014. "Identifying the Confounders of Marginalization and Mortality in Mexico, 2003–2007," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 118(2), pages 851-875, September.
    14. A. Goubar & A. E. Ades & D. De Angelis & C. A. McGarrigle & C. H. Mercer & P. A. Tookey & K. Fenton & O. N. Gill, 2008. "Estimates of human immunodeficiency virus prevalence and proportion diagnosed based on Bayesian multiparameter synthesis of surveillance data," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 171(3), pages 541-580, June.
    15. Maria Gheorghe & Susan Picavet & Monique Verschuren & Werner B. F. Brouwer & Pieter H. M. Baal, 2017. "Health losses at the end of life: a Bayesian mixed beta regression approach," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 180(3), pages 723-749, June.
    16. Andrew Gelman & Christian Hennig, 2017. "Beyond subjective and objective in statistics," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 180(4), pages 967-1033, October.
    17. Rebecca M. Turner & David J. Spiegelhalter & Gordon C. S. Smith & Simon G. Thompson, 2009. "Bias modelling in evidence synthesis," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 172(1), pages 21-47, January.
    18. Newmark Armstrong, Georgina & Lairson, David R., 2006. "Cost-effectiveness of alternate contact protocols and costs of mammography promotion interventions for women veterans," Evaluation and Program Planning, Elsevier, vol. 29(2), pages 120-129, May.
    19. Qi Zhou & Yoo-Mi Chin & James D. Stamey & Joon Jin Song, 2020. "Bayesian sensitivity analysis to unmeasured confounding for misclassified data," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 104(4), pages 577-596, December.
    20. Giancarlo MANZI & Pier Alda FERRARI, "undated". "Statistical methods for evaluating satisfaction with public services Abstract: Contrary to private enterprises, public enterprises can be unaware of the impact of their performance when providing serv," CIRIEC Working Papers 1404, CIRIEC - Université de Liège.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssa:v:183:y:2020:i:1:p:193-209. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/rssssea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.