IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0163309.html
   My bibliography  Save this article

Reproducibility of Search Strategies Is Poor in Systematic Reviews Published in High-Impact Pediatrics, Cardiology and Surgery Journals: A Cross-Sectional Study

Author

Listed:
  • Jonathan B Koffel
  • Melissa L Rethlefsen

Abstract

Background: A high-quality search strategy is considered an essential component of systematic reviews but many do not contain reproducible search strategies. It is unclear if low reproducibility spans medical disciplines, is affected by librarian/search specialist involvement or has improved with increased awareness of reporting guidelines. Objectives: To examine the reporting of search strategies in systematic reviews published in Pediatrics, Surgery or Cardiology journals in 2012 and determine rates and predictors of including a reproducible search strategy. Methods: We identified all systematic reviews published in 2012 in the ten highest impact factor journals in Pediatrics, Surgery and Cardiology. Each search strategy was coded to indicate what elements were reported and whether the overall search was reproducible. Reporting and reproducibility rates were compared across disciplines and we measured the influence of librarian/search specialist involvement, discipline or endorsement of a reporting guideline on search reproducibility. Results: 272 articles from 25 journals were included. Reporting of search elements ranged widely from 91% of articles naming search terms to 33% providing a full search strategy and 22% indicating the date the search was executed. Only 22% of articles provided at least one reproducible search strategy and 13% provided a reproducible strategy for all databases searched in the article. Librarians or search specialists were reported as involved in 17% of articles. There were strong disciplinary differences on the reporting of search elements. In the multivariable analysis, only discipline (Pediatrics) was a significant predictor of the inclusion of a reproducible search strategy. Conclusions: Despite recommendations to report full, reproducible search strategies, many articles still do not. In addition, authors often report a single strategy as covering all databases searched, further decreasing reproducibility. Further research is needed to determine how disciplinary culture may encourage reproducibility and the role that journal editors and peer reviewers could play.

Suggested Citation

  • Jonathan B Koffel & Melissa L Rethlefsen, 2016. "Reproducibility of Search Strategies Is Poor in Systematic Reviews Published in High-Impact Pediatrics, Cardiology and Surgery Journals: A Cross-Sectional Study," PLOS ONE, Public Library of Science, vol. 11(9), pages 1-16, September.
    • Handle: RePEc:plo:pone00:0163309
    DOI: 10.1371/journal.pone.0163309
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0163309
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0163309&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0163309?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Lucy Turner & James Galipeau & Chantelle Garritty & Eric Manheimer & L Susan Wieland & Fatemeh Yazdi & David Moher, 2013. "An Evaluation of Epidemiological and Reporting Characteristics of Complementary and Alternative Medicine (CAM) Systematic Reviews (SRs)," PLOS ONE, Public Library of Science, vol. 8(1), pages 1-11, January.
    2. Long Ge & Jian-cheng Wang & Jin-long Li & Li Liang & Ni An & Xin-tong Shi & Yin-chun Liu & Jin-hui Tian, 2014. "The Assessment of the Quality of Reporting of Systematic Reviews/Meta-Analyses in Diagnostic Tests Published by Authors in China," PLOS ONE, Public Library of Science, vol. 9(1), pages 1-11, January.
    3. Brian Hutton & Georgia Salanti & Anna Chaimani & Deborah M Caldwell & Chris Schmid & Kristian Thorlund & Edward Mills & Ferrán Catalá-López & Lucy Turner & Douglas G Altman & David Moher, 2014. "The Quality of Reporting Methods and Results in Network Meta-Analyses: An Overview of Reviews and Suggestions for Improvement," PLOS ONE, Public Library of Science, vol. 9(3), pages 1-9, March.
    4. Padhraig S Fleming & Despina Koletsi & Nikolaos Pandis, 2014. "Blinded by PRISMA: Are Systematic Reviewers Focusing on PRISMA and Ignoring Other Guidelines?," PLOS ONE, Public Library of Science, vol. 9(5), pages 1-7, May.
    5. Jeroen P M Peters & Lotty Hooft & Wilko Grolman & Inge Stegeman, 2015. "Reporting Quality of Systematic Reviews and Meta-Analyses of Otorhinolaryngologic Articles Based on the PRISMA Statement," PLOS ONE, Public Library of Science, vol. 10(8), pages 1-11, August.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Fernanda S Tonin & Laiza M Steimbach & Antonio M Mendes & Helena H Borba & Roberto Pontarolo & Fernando Fernandez-Llimos, 2018. "Mapping the characteristics of network meta-analyses on drug therapy: A systematic review," PLOS ONE, Public Library of Science, vol. 13(4), pages 1-16, April.
    2. Seo, Chunghyeon & Kim, Bitna & Kruis, Nathan E., 2021. "Variation across police response models for handling encounters with people with mental illnesses: A systematic review and meta-analysis," Journal of Criminal Justice, Elsevier, vol. 72(C).
    3. Despina Koletsi & Anna Iliadi & Giorgos N. Tzanetakis & Manolis Vavuranakis & Theodore Eliades, 2021. "Cardiovascular Disease and Chronic Endodontic Infection. Is There an Association? A Systematic Review and Meta-Analysis," IJERPH, MDPI, vol. 18(17), pages 1-18, August.
    4. Ferrán Catalá-López & Brian Hutton & Amparo Núñez-Beltrán & Matthew J Page & Manuel Ridao & Diego Macías Saint-Gerons & Miguel A Catalá & Rafael Tabarés-Seisdedos & David Moher, 2017. "The pharmacological and non-pharmacological treatment of attention deficit hyperactivity disorder in children and adolescents: A systematic review with network meta-analyses of randomised trials," PLOS ONE, Public Library of Science, vol. 12(7), pages 1-31, July.
    5. Kim, Bitna, 2022. "Publication bias: A “bird's-eye view” of meta-analytic practice in criminology and criminal justice," Journal of Criminal Justice, Elsevier, vol. 78(C).
    6. Jonathan B Koffel, 2015. "Use of Recommended Search Strategies in Systematic Reviews and the Impact of Librarian Involvement: A Cross-Sectional Survey of Recent Authors," PLOS ONE, Public Library of Science, vol. 10(5), pages 1-13, May.
    7. Emilio Abad-Segura & Mariana-Daniela González-Zamar & Antonio Luque-de la Rosa & María Belén Morales Cevallos, 2020. "Sustainability of Educational Technologies: An Approach to Augmented Reality Research," Sustainability, MDPI, vol. 12(10), pages 1-28, May.
    8. Hisham Abusaada & Abeer Elshater, 2022. "Notes on Developing Research Review in Urban Planning and Urban Design Based on PRISMA Statement," Social Sciences, MDPI, vol. 11(9), pages 1-8, August.
    9. Marialuisa Saviano & Sergio Barile & Francesco Caputo & Mattia Lettieri & Stefania Zanda, 2019. "From Rare to Neglected Diseases: A Sustainable and Inclusive Healthcare Perspective for Reframing the Orphan Drugs Issue," Sustainability, MDPI, vol. 11(5), pages 1-21, March.
    10. Abeer Elshater & Hisham Abusaada, 2022. "Developing Process for Selecting Research Techniques in Urban Planning and Urban Design with a PRISMA-Compliant Review," Social Sciences, MDPI, vol. 11(10), pages 1-17, October.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0163309. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.