IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0227580.html
   My bibliography  Save this article

Assessment of publication bias and outcome reporting bias in systematic reviews of health services and delivery research: A meta-epidemiological study

Author

Listed:
  • Abimbola A Ayorinde
  • Iestyn Williams
  • Russell Mannion
  • Fujian Song
  • Magdalena Skrybant
  • Richard J Lilford
  • Yen-Fu Chen

Abstract

Strategies to identify and mitigate publication bias and outcome reporting bias are frequently adopted in systematic reviews of clinical interventions but it is not clear how often these are applied in systematic reviews relating to quantitative health services and delivery research (HSDR). We examined whether these biases are mentioned and/or otherwise assessed in HSDR systematic reviews, and evaluated associating factors to inform future practice. We randomly selected 200 quantitative HSDR systematic reviews published in the English language from 2007–2017 from the Health Systems Evidence database (www.healthsystemsevidence.org). We extracted data on factors that may influence whether or not authors mention and/or assess publication bias or outcome reporting bias. We found that 43% (n = 85) of the reviews mentioned publication bias and 10% (n = 19) formally assessed it. Outcome reporting bias was mentioned and assessed in 17% (n = 34) of all the systematic reviews. Insufficient number of studies, heterogeneity and lack of pre-registered protocols were the most commonly reported impediments to assessing the biases. In multivariable logistic regression models, both mentioning and formal assessment of publication bias were associated with: inclusion of a meta-analysis; being a review of intervention rather than association studies; higher journal impact factor, and; reporting the use of systematic review guidelines. Assessment of outcome reporting bias was associated with: being an intervention review; authors reporting the use of Grading of Recommendations, Assessment, Development and Evaluations (GRADE), and; inclusion of only controlled trials. Publication bias and outcome reporting bias are infrequently assessed in HSDR systematic reviews. This may reflect the inherent heterogeneity of HSDR evidence and different methodological approaches to synthesising the evidence, lack of awareness of such biases, limits of current tools and lack of pre-registered study protocols for assessing such biases. Strategies to help raise awareness of the biases, and methods to minimise their occurrence and mitigate their impacts on HSDR systematic reviews, are needed.

Suggested Citation

  • Abimbola A Ayorinde & Iestyn Williams & Russell Mannion & Fujian Song & Magdalena Skrybant & Richard J Lilford & Yen-Fu Chen, 2020. "Assessment of publication bias and outcome reporting bias in systematic reviews of health services and delivery research: A meta-epidemiological study," PLOS ONE, Public Library of Science, vol. 15(1), pages 1-17, January.
    • Handle: RePEc:plo:pone00:0227580
    DOI: 10.1371/journal.pone.0227580
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0227580
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0227580&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0227580?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Leroy, J.L. & Habicht, J.-P. & Pelto, G. & Bertozzi, S.M., 2007. "Current priorities in health research funding and lack of impact on the number of child deaths per year," American Journal of Public Health, American Public Health Association, vol. 97(2), pages 219-223.
    2. Sara Bennett & Irene Akua Agyepong & Kabir Sheikh & Kara Hanson & Freddie Ssengooba & Lucy Gilson, 2011. "Building the Field of Health Policy and Systems Research: An Agenda for Action," PLOS Medicine, Public Library of Science, vol. 8(8), pages 1-5, August.
    3. Matthew J Page & Larissa Shamseer & Douglas G Altman & Jennifer Tetzlaff & Margaret Sampson & Andrea C Tricco & Ferrán Catalá-López & Lun Li & Emma K Reid & Rafael Sarkis-Onofre & David Moher, 2016. "Epidemiology and Reporting Characteristics of Systematic Reviews of Biomedical Research: A Cross-Sectional Study," PLOS Medicine, Public Library of Science, vol. 13(5), pages 1-30, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Mangirdas Morkunas & Elzė Rudienė & Lukas Giriūnas & Laura Daučiūnienė, 2020. "Assessment of Factors Causing Bias in Marketing- Related Publications," Publications, MDPI, vol. 8(4), pages 1-16, October.
    2. Julia Heffernan & Ewan McDonald & Elizabeth Hughes & Richard Gray, 2021. "Tri-Response Police, Ambulance, Mental Health Crisis Models in Reducing Involuntary Detentions of Mentally Ill People: Protocol for a Systematic Review," IJERPH, MDPI, vol. 18(15), pages 1-9, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Neal R. Haddaway & Matthew J. Page & Chris C. Pritchard & Luke A. McGuinness, 2022. "PRISMA2020: An R package and Shiny app for producing PRISMA 2020‐compliant flow diagrams, with interactivity for optimised digital transparency and Open Synthesis," Campbell Systematic Reviews, John Wiley & Sons, vol. 18(2), June.
    2. Camelia Lucia Bakri & Maria Daniela Pipas, 2019. "Performance Management and Quality Assurance in Primary Healthcare Institutions," Proceedings of the 12th International RAIS Conference, April 3-4, 2019 22BC, Research Association for Interdisciplinary Studies.
    3. Lenka Benova & Ann-Beth Moller & Allisyn C Moran, 2019. "“What gets measured better gets done better”: The landscape of validation of global maternal and newborn health indicators through key informant interviews," PLOS ONE, Public Library of Science, vol. 14(11), pages 1-16, November.
    4. Kathleen Van Royen & Carl Lachat & Michelle Holdsworth & Karlien Smit & Joyce Kinabo & Dominique Roberfroid & Eunice Nago & Christopher Garimoi Orach & Patrick Kolsteren, 2013. "How Can the Operating Environment for Nutrition Research Be Improved in Sub-Saharan Africa? The Views of African Researchers," PLOS ONE, Public Library of Science, vol. 8(6), pages 1-9, June.
    5. Josephine Borghi & Garrett W. Brown, 2022. "Taking Systems Thinking to the Global Level: Using the WHO Building Blocks to Describe and Appraise the Global Health System in Relation to COVID‐19," Global Policy, London School of Economics and Political Science, vol. 13(2), pages 193-207, May.
    6. Taufique Joarder & Asha George & Syed Masud Ahmed & Sabina Faiz Rashid & Malabika Sarker, 2017. "What constitutes responsiveness of physicians: A qualitative study in rural Bangladesh," PLOS ONE, Public Library of Science, vol. 12(12), pages 1-19, December.
    7. Nargiz Travis & Marie Knoll & Christopher J. Cadham & Steven Cook & Kenneth E. Warner & Nancy L. Fleischer & Clifford E. Douglas & Luz María Sánchez-Romero & Ritesh Mistry & Rafael Meza & Jana L. Hirs, 2022. "Health Effects of Electronic Cigarettes: An Umbrella Review and Methodological Considerations," IJERPH, MDPI, vol. 19(15), pages 1-23, July.
    8. Taghreed Adam & Saad Ahmad & Maryam Bigdeli & Abdul Ghaffar & John-Arne Røttingen, 2011. "Trends in Health Policy and Systems Research over the Past Decade: Still Too Little Capacity in Low-Income Countries," PLOS ONE, Public Library of Science, vol. 6(11), pages 1-10, November.
    9. Rockers, Peter C. & Røttingen, John-Arne & Shemilt, Ian & Tugwell, Peter & Bärnighausen, Till, 2015. "Inclusion of quasi-experimental studies in systematic reviews of health systems research," Health Policy, Elsevier, vol. 119(4), pages 511-521.
    10. Matthew J Page & Joanne E McKenzie & Patrick M Bossuyt & Isabelle Boutron & Tammy C Hoffmann & Cynthia D Mulrow & Larissa Shamseer & Jennifer M Tetzlaff & Elie A Akl & Sue E Brennan & Roger Chou & Jul, 2021. "The PRISMA 2020 statement: An updated guideline for reporting systematic reviews," PLOS Medicine, Public Library of Science, vol. 18(3), pages 1-15, March.
    11. Edwine W Barasa & Anthony M Manyara & Sassy Molyneux & Benjamin Tsofa, 2017. "Recentralization within decentralization: County hospital autonomy under devolution in Kenya," PLOS ONE, Public Library of Science, vol. 12(8), pages 1-18, August.
    12. Margaret E Kruk & Gavin Yamey & Sonia Y Angell & Alix Beith & Daniel Cotlear & Frederico Guanais & Lisa Jacobs & Helen Saxenian & Cesar Victora & Eric Goosby, 2016. "Transforming Global Health by Improving the Science of Scale-Up," PLOS Biology, Public Library of Science, vol. 14(3), pages 1-16, March.
    13. Masahiro Banno & Yasushi Tsujimoto & Kunihiro Kohmura & Eisuke Dohi & Shunsuke Taito & Hidehiro Someko & Yuki Kataoka, 2022. "Unclear Insomnia Concept in Randomized Controlled Trials and Systematic Reviews: A Meta-Epidemiological Study," IJERPH, MDPI, vol. 19(19), pages 1-10, September.
    14. Shilpa Surendran & Chuan De Foo & Chen Hee Tam & Elaine Qiao Ying Ho & David Bruce Matchar & Josip Car & Gerald Choon Huat Koh, 2021. "The Missed Opportunity of Patient-Centered Medical Homes to Thrive in an Asian Context," IJERPH, MDPI, vol. 18(4), pages 1-11, February.
    15. Ricardo B. Sampaio & Bruna P. F. Fonseca & Ashwin Bahulkar & Boleslaw K. Szymanski, 2017. "Network analysis to support public health: evolution of collaboration among leishmaniasis researchers," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(3), pages 2001-2021, June.
    16. Hutchinson, Eleanor & Mundua, Sunday & Ochero, Lydia & Mbonye, Anthony & Clarke, Sian E., 2022. "Life in the buffer zone: Social relations and surplus health workers in Uganda's medicines retail sector," Social Science & Medicine, Elsevier, vol. 300(C).
    17. Sadri, Arash, 2022. "Machine Learning Can Solve the Reproducibility Crisis by Supplanting Reductionist Statistics," MetaArXiv yxba5_v1, Center for Open Science.
    18. Nargiz Travis & Marie Knoll & Steven Cook & Hayoung Oh & Christopher J. Cadham & Luz María Sánchez-Romero & David T. Levy, 2023. "Chemical Profiles and Toxicity of Electronic Cigarettes: An Umbrella Review and Methodological Considerations," IJERPH, MDPI, vol. 20(3), pages 1-15, January.
    19. Bodil Hoffmeyer & Siv Fonnes & Kristoffer Andresen & Jacob Rosenberg, 2023. "Use of inactive Cochrane reviews in academia: A citation analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(5), pages 2923-2934, May.
    20. Coburn, Josie & Yaqub, Ohid & Chataway, Joanna, 2022. "Targeting research to address societal needs: What can we learn from 30 years of targeting neglected diseases?," SocArXiv 65ws7, Center for Open Science.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0227580. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.