IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0025491.html
   My bibliography  Save this article

The Number of Patients and Events Required to Limit the Risk of Overestimation of Intervention Effects in Meta-Analysis—A Simulation Study

Author

Listed:
  • Kristian Thorlund
  • Georgina Imberger
  • Michael Walsh
  • Rong Chu
  • Christian Gluud
  • Jørn Wetterslev
  • Gordon Guyatt
  • Philip J Devereaux
  • Lehana Thabane

Abstract

Background: Meta-analyses including a limited number of patients and events are prone to yield overestimated intervention effect estimates. While many assume bias is the cause of overestimation, theoretical considerations suggest that random error may be an equal or more frequent cause. The independent impact of random error on meta-analyzed intervention effects has not previously been explored. It has been suggested that surpassing the optimal information size (i.e., the required meta-analysis sample size) provides sufficient protection against overestimation due to random error, but this claim has not yet been validated. Methods: We simulated a comprehensive array of meta-analysis scenarios where no intervention effect existed (i.e., relative risk reduction (RRR) = 0%) or where a small but possibly unimportant effect existed (RRR = 10%). We constructed different scenarios by varying the control group risk, the degree of heterogeneity, and the distribution of trial sample sizes. For each scenario, we calculated the probability of observing overestimates of RRR>20% and RRR>30% for each cumulative 500 patients and 50 events. We calculated the cumulative number of patients and events required to reduce the probability of overestimation of intervention effect to 10%, 5%, and 1%. We calculated the optimal information size for each of the simulated scenarios and explored whether meta-analyses that surpassed their optimal information size had sufficient protection against overestimation of intervention effects due to random error. Results: The risk of overestimation of intervention effects was usually high when the number of patients and events was small and this risk decreased exponentially over time as the number of patients and events increased. The number of patients and events required to limit the risk of overestimation depended considerably on the underlying simulation settings. Surpassing the optimal information size generally provided sufficient protection against overestimation. Conclusions: Random errors are a frequent cause of overestimation of intervention effects in meta-analyses. Surpassing the optimal information size will provide sufficient protection against overestimation.

Suggested Citation

  • Kristian Thorlund & Georgina Imberger & Michael Walsh & Rong Chu & Christian Gluud & Jørn Wetterslev & Gordon Guyatt & Philip J Devereaux & Lehana Thabane, 2011. "The Number of Patients and Events Required to Limit the Risk of Overestimation of Intervention Effects in Meta-Analysis—A Simulation Study," PLOS ONE, Public Library of Science, vol. 6(10), pages 1-10, October.
  • Handle: RePEc:plo:pone00:0025491
    DOI: 10.1371/journal.pone.0025491
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0025491
    Download Restriction: no

    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0025491&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pone.0025491?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. James G. Field & Frank A. Bosco & David Kraichy & Krista L. Uggerslev & Mingang K. Geiger, 2021. "More alike than different? A comparison of variance explained by cross-cultural models," Journal of International Business Studies, Palgrave Macmillan;Academy of International Business, vol. 52(9), pages 1797-1817, December.
    2. Sheng Li & Xian-Tao Zeng & Xiao-Lan Ruan & Hong Weng & Tong-Zu Liu & Xiao Wang & Chao Zhang & Zhe Meng & Xing-Huan Wang, 2014. "Holmium Laser Enucleation versus Transurethral Resection in Patients with Benign Prostate Hyperplasia: An Updated Systematic Review with Meta-Analysis and Trial Sequential Analysis," PLOS ONE, Public Library of Science, vol. 9(7), pages 1-14, July.
    3. Cho-Hao Lee & Jung-Chung Lin & Ching-Liang Ho & Min Sun & Wel-Ting Yen & Chin Lin, 2017. "Efficacy and safety of micafungin versus extensive azoles in the prevention and treatment of invasive fungal infections for neutropenia patients with hematological malignancies: A meta-analysis of ran," PLOS ONE, Public Library of Science, vol. 12(7), pages 1-20, July.
    4. Kulinskaya, Elena & Mah, Eung Yaw, 2021. "Simulation results on the performance of statistical methods in cumulative meta analysis," MetaArXiv 8t4pf, Center for Open Science.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0025491. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.