IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0227580.html
   My bibliography  Save this article

Assessment of publication bias and outcome reporting bias in systematic reviews of health services and delivery research: A meta-epidemiological study

Author

Listed:
  • Abimbola A Ayorinde
  • Iestyn Williams
  • Russell Mannion
  • Fujian Song
  • Magdalena Skrybant
  • Richard J Lilford
  • Yen-Fu Chen

Abstract

Strategies to identify and mitigate publication bias and outcome reporting bias are frequently adopted in systematic reviews of clinical interventions but it is not clear how often these are applied in systematic reviews relating to quantitative health services and delivery research (HSDR). We examined whether these biases are mentioned and/or otherwise assessed in HSDR systematic reviews, and evaluated associating factors to inform future practice. We randomly selected 200 quantitative HSDR systematic reviews published in the English language from 2007–2017 from the Health Systems Evidence database (www.healthsystemsevidence.org). We extracted data on factors that may influence whether or not authors mention and/or assess publication bias or outcome reporting bias. We found that 43% (n = 85) of the reviews mentioned publication bias and 10% (n = 19) formally assessed it. Outcome reporting bias was mentioned and assessed in 17% (n = 34) of all the systematic reviews. Insufficient number of studies, heterogeneity and lack of pre-registered protocols were the most commonly reported impediments to assessing the biases. In multivariable logistic regression models, both mentioning and formal assessment of publication bias were associated with: inclusion of a meta-analysis; being a review of intervention rather than association studies; higher journal impact factor, and; reporting the use of systematic review guidelines. Assessment of outcome reporting bias was associated with: being an intervention review; authors reporting the use of Grading of Recommendations, Assessment, Development and Evaluations (GRADE), and; inclusion of only controlled trials. Publication bias and outcome reporting bias are infrequently assessed in HSDR systematic reviews. This may reflect the inherent heterogeneity of HSDR evidence and different methodological approaches to synthesising the evidence, lack of awareness of such biases, limits of current tools and lack of pre-registered study protocols for assessing such biases. Strategies to help raise awareness of the biases, and methods to minimise their occurrence and mitigate their impacts on HSDR systematic reviews, are needed.

Suggested Citation

  • Abimbola A Ayorinde & Iestyn Williams & Russell Mannion & Fujian Song & Magdalena Skrybant & Richard J Lilford & Yen-Fu Chen, 2020. "Assessment of publication bias and outcome reporting bias in systematic reviews of health services and delivery research: A meta-epidemiological study," PLOS ONE, Public Library of Science, vol. 15(1), pages 1-17, January.
    • Handle: RePEc:plo:pone00:0227580
    DOI: 10.1371/journal.pone.0227580
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0227580
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0227580&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0227580?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Matthew J Page & Larissa Shamseer & Douglas G Altman & Jennifer Tetzlaff & Margaret Sampson & Andrea C Tricco & Ferrán Catalá-López & Lun Li & Emma K Reid & Rafael Sarkis-Onofre & David Moher, 2016. "Epidemiology and Reporting Characteristics of Systematic Reviews of Biomedical Research: A Cross-Sectional Study," PLOS Medicine, Public Library of Science, vol. 13(5), pages 1-30, May.
    2. Leroy, J.L. & Habicht, J.-P. & Pelto, G. & Bertozzi, S.M., 2007. "Current priorities in health research funding and lack of impact on the number of child deaths per year," American Journal of Public Health, American Public Health Association, vol. 97(2), pages 219-223.
    3. Sara Bennett & Irene Akua Agyepong & Kabir Sheikh & Kara Hanson & Freddie Ssengooba & Lucy Gilson, 2011. "Building the Field of Health Policy and Systems Research: An Agenda for Action," PLOS Medicine, Public Library of Science, vol. 8(8), pages 1-5, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Julia Heffernan & Ewan McDonald & Elizabeth Hughes & Richard Gray, 2021. "Tri-Response Police, Ambulance, Mental Health Crisis Models in Reducing Involuntary Detentions of Mentally Ill People: Protocol for a Systematic Review," IJERPH, MDPI, vol. 18(15), pages 1-9, August.
    2. Mangirdas Morkunas & Elzė Rudienė & Lukas Giriūnas & Laura Daučiūnienė, 2020. "Assessment of Factors Causing Bias in Marketing- Related Publications," Publications, MDPI, vol. 8(4), pages 1-16, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Neal R. Haddaway & Matthew J. Page & Chris C. Pritchard & Luke A. McGuinness, 2022. "PRISMA2020: An R package and Shiny app for producing PRISMA 2020‐compliant flow diagrams, with interactivity for optimised digital transparency and Open Synthesis," Campbell Systematic Reviews, John Wiley & Sons, vol. 18(2), June.
    2. Meena Daivadanam & Maia Ingram & Kristi Sidney Annerstedt & Gary Parker & Kirsty Bobrow & Lisa Dolovich & Gillian Gould & Michaela Riddell & Rajesh Vedanthan & Jacqui Webster & Pilvikki Absetz & Helle, 2019. "The role of context in implementation research for non-communicable diseases: Answering the ‘how-to’ dilemma," PLOS ONE, Public Library of Science, vol. 14(4), pages 1-22, April.
    3. Topp, Stephanie M. & Tully, Josslyn & Cummins, Rachel & Graham, Veronica & Yashadhana, Aryati & Elliott, Lana & Taylor, Sean, 2022. "Building patient trust in health systems: A qualitative study of facework in the context of the Aboriginal and Torres Strait Islander Health Worker role in Queensland, Australia," Social Science & Medicine, Elsevier, vol. 302(C).
    4. Camelia Lucia Bakri & Maria Daniela Pipas, 2019. "Performance Management and Quality Assurance in Primary Healthcare Institutions," Proceedings of the 12th International RAIS Conference, April 3-4, 2019 22BC, Research Association for Interdisciplinary Studies.
    5. Lenka Benova & Ann-Beth Moller & Allisyn C Moran, 2019. "“What gets measured better gets done better”: The landscape of validation of global maternal and newborn health indicators through key informant interviews," PLOS ONE, Public Library of Science, vol. 14(11), pages 1-16, November.
    6. Kathleen Van Royen & Carl Lachat & Michelle Holdsworth & Karlien Smit & Joyce Kinabo & Dominique Roberfroid & Eunice Nago & Christopher Garimoi Orach & Patrick Kolsteren, 2013. "How Can the Operating Environment for Nutrition Research Be Improved in Sub-Saharan Africa? The Views of African Researchers," PLOS ONE, Public Library of Science, vol. 8(6), pages 1-9, June.
    7. Josephine Borghi & Garrett W. Brown, 2022. "Taking Systems Thinking to the Global Level: Using the WHO Building Blocks to Describe and Appraise the Global Health System in Relation to COVID‐19," Global Policy, London School of Economics and Political Science, vol. 13(2), pages 193-207, May.
    8. Taufique Joarder & Asha George & Syed Masud Ahmed & Sabina Faiz Rashid & Malabika Sarker, 2017. "What constitutes responsiveness of physicians: A qualitative study in rural Bangladesh," PLOS ONE, Public Library of Science, vol. 12(12), pages 1-19, December.
    9. Nargiz Travis & Marie Knoll & Christopher J. Cadham & Steven Cook & Kenneth E. Warner & Nancy L. Fleischer & Clifford E. Douglas & Luz María Sánchez-Romero & Ritesh Mistry & Rafael Meza & Jana L. Hirs, 2022. "Health Effects of Electronic Cigarettes: An Umbrella Review and Methodological Considerations," IJERPH, MDPI, vol. 19(15), pages 1-23, July.
    10. Magrath, Priscilla & Nichter, Mark, 2012. "Paying for performance and the social relations of health care provision: An anthropological perspective," Social Science & Medicine, Elsevier, vol. 75(10), pages 1778-1785.
    11. Closser, Svea & Mendenhall, Emily & Brown, Peter & Neill, Rachel & Justice, Judith, 2022. "The anthropology of health systems: A history and review," Social Science & Medicine, Elsevier, vol. 300(C).
    12. Taghreed Adam & Saad Ahmad & Maryam Bigdeli & Abdul Ghaffar & John-Arne Røttingen, 2011. "Trends in Health Policy and Systems Research over the Past Decade: Still Too Little Capacity in Low-Income Countries," PLOS ONE, Public Library of Science, vol. 6(11), pages 1-10, November.
    13. Colvin, Christopher J., 2022. "Understanding global health policy engagements with qualitative research: Qualitative evidence syntheses and the OptimizeMNH guidelines," Social Science & Medicine, Elsevier, vol. 300(C).
    14. Claudia Nieto-Sanchez & Benjamin R Bates & Darwin Guerrero & Sylvia Jimenez & Esteban G Baus & Koen Peeters Grietens & Mario J Grijalva, 2019. "Home improvement and system-based health promotion for sustainable prevention of Chagas disease: A qualitative study," PLOS Neglected Tropical Diseases, Public Library of Science, vol. 13(6), pages 1-24, June.
    15. Taghreed El Hajj & Stefanie Gregorius & Justin Pulford & Imelda Bates, 2020. "Strengthening capacity for natural sciences research: A qualitative assessment to identify good practices, capacity gaps and investment priorities in African research institutions," PLOS ONE, Public Library of Science, vol. 15(1), pages 1-21, January.
    16. Esther Maassen & Marcel A L M van Assen & Michèle B Nuijten & Anton Olsson-Collentine & Jelte M Wicherts, 2020. "Reproducibility of individual effect sizes in meta-analyses in psychology," PLOS ONE, Public Library of Science, vol. 15(5), pages 1-18, May.
    17. Rockers, Peter C. & Røttingen, John-Arne & Shemilt, Ian & Tugwell, Peter & Bärnighausen, Till, 2015. "Inclusion of quasi-experimental studies in systematic reviews of health systems research," Health Policy, Elsevier, vol. 119(4), pages 511-521.
    18. Matthew J Page & Joanne E McKenzie & Patrick M Bossuyt & Isabelle Boutron & Tammy C Hoffmann & Cynthia D Mulrow & Larissa Shamseer & Jennifer M Tetzlaff & Elie A Akl & Sue E Brennan & Roger Chou & Jul, 2021. "The PRISMA 2020 statement: An updated guideline for reporting systematic reviews," PLOS Medicine, Public Library of Science, vol. 18(3), pages 1-15, March.
    19. Edwine W Barasa & Anthony M Manyara & Sassy Molyneux & Benjamin Tsofa, 2017. "Recentralization within decentralization: County hospital autonomy under devolution in Kenya," PLOS ONE, Public Library of Science, vol. 12(8), pages 1-18, August.
    20. Mladovsky, Philipa, 2020. "Fragmentation by design: Universal health coverage policies as governmentality in Senegal," Social Science & Medicine, Elsevier, vol. 260(C).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0227580. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.