IDEAS home Printed from https://ideas.repec.org/a/wly/camsys/v20y2024i3ne1432.html
   My bibliography  Save this article

Searching and reporting in Campbell Collaboration systematic reviews: A systematic assessment of current methods

Author

Listed:
  • Sarah Young
  • Heather MacDonald
  • Diana Louden
  • Ursula M. Ellis
  • Zahra Premji
  • Morwenna Rogers
  • Alison Bethel
  • David Pickup

Abstract

The search methods used in systematic reviews provide the foundation for establishing the body of literature from which conclusions are drawn and recommendations made. Searches should aim to be comprehensive and reporting of search methods should be transparent and reproducible. Campbell Collaboration systematic reviews strive to adhere to the best methodological guidance available for this type of searching. The current work aims to provide an assessment of the conduct and reporting of searches in Campbell Collaboration systematic reviews. Our objectives were to examine how searches are currently conducted in Campbell systematic reviews, how search strategies, search methods and search reporting adhere to the Methodological Expectations of Campbell Collaboration Intervention Reviews (MECCIR) and PRISMA standards, and identify emerging or novel methods used in searching in Campbell systematic reviews. We also investigated the role of information specialists in Campbell systematic reviews. We handsearched the Campbell Systematic Reviews journal tables of contents from January 2017 to March 2024. We included all systematic reviews published since 2017. We excluded other types of evidence synthesis (e.g., evidence and gap maps), updates to systematic reviews when search methods were not changed from the original pre‐2017 review, and systematic reviews that did not conduct their own original searches. We developed a data extraction form in part based on the conduct and reporting items in MECCIR and PRISMA. In addition, we extracted information about the general quality of searches based on the use of Boolean operators, keywords, database syntax and subject headings. Data extraction included information about reporting of sources searched, some aspects of search quality, the use and reporting of supplementary search methods, reporting of the search strategy, the involvement of information specialists, date of the most recent search, and citation of the Campbell search methods guidance. Items were rated as fully, partially or not conducted or reported. We cross‐walked our data extraction items to the 2019 MECCIR standards and 2020 PRISMA guidelines and provide descriptive analyses of the conduct and reporting of searches in Campbell systematic reviews, indicating level of adherence to standards where applicable. We included 111 Campbell systematic reviews across all coordinating groups published since 2017 up to the search date. Almost all (98%) included reviews searched at least two relevant databases and all reported the databases searched. All reviews searched grey literature and most (82%) provided a full list of grey literature sources. Detailed information about databases such as platform and date range coverage was lacking in 16% and 77% of the reviews, respectively. In terms of search strategies, most used Boolean operators, search syntax and phrase searching correctly, but subject headings in databases with controlled vocabulary were used in only about half of the reviews. Most reviews reported at least one full database search strategy (90%), with 63% providing full search strategies for all databases. Most reviews conducted some supplementary searching, most commonly searching the references of included studies, whereas handsearching of journals and forward citation searching were less commonly reported (51% and 62%, respectively). Twenty‐nine percent of reviews involved an information specialist co‐author and about 45% did not mention the involvement of any information specialist. When information specialists were co‐authors, there was a concomitant increase in adherence to many reporting and conduct standards and guidelines, including reporting website URLs, reporting methods for forward citation searching, using database syntax correctly and using subject headings. No longitudinal trends in adherence to conducting and reporting standards were found and the Campbell search methods guidance published in 2017 was cited in only twelve reviews. We also found a median time lag of 20 months between the most recent search and the publication date. In general, the included Campbell systematic reviews searched a wide range of bibliographic databases and grey literature, and conducted at least some supplementary searching such as searching references of included studies or contacting experts. Reporting of mandatory standards was variable with some frequently unreported (e.g., website URLs and database date ranges) and others well reported in most reviews. For example, database search strategies were reported in detail in most reviews. For grey literature, source names were well reported but search strategies were less so. The findings will be used to identify opportunities for advancing current practices in Campbell reviews through updated guidance, peer review processes and author training and support.

Suggested Citation

  • Sarah Young & Heather MacDonald & Diana Louden & Ursula M. Ellis & Zahra Premji & Morwenna Rogers & Alison Bethel & David Pickup, 2024. "Searching and reporting in Campbell Collaboration systematic reviews: A systematic assessment of current methods," Campbell Systematic Reviews, John Wiley & Sons, vol. 20(3), September.
  • Handle: RePEc:wly:camsys:v:20:y:2024:i:3:n:e1432
    DOI: 10.1002/cl2.1432
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/cl2.1432
    Download Restriction: no

    File URL: https://libkey.io/10.1002/cl2.1432?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Shannon Kugley & Anne Wade & James Thomas & Quenby Mahood & Anne‐Marie Klint Jørgensen & Karianne Hammerstrøm & Nila Sathe, 2017. "Searching for studies: a guide to information retrieval for Campbell systematic reviews," Campbell Systematic Reviews, John Wiley & Sons, vol. 13(1), pages 1-73.
    2. Xiaoqin Wang & Vivian Welch & Meixuan Li & Liang Yao & Julia Littell & Huijuan Li & Nan Yang & Jianjian Wang & Larissa Shamseer & Yaolong Chen & Kehu Yang & Jeremy M. Grimshaw, 2021. "The methodological and reporting characteristics of Campbell reviews: A systematic review," Campbell Systematic Reviews, John Wiley & Sons, vol. 17(1), March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sarah Young & Alison Bethel & Ciara Keenan & Kate Ghezzi‐Kopel & Elizabeth Moreton & David Pickup & Zahra A. Premji & Morwenna Rogers & Bjørn C. A. Viinholt, 2021. "PROTOCOL: Searching and reporting in Campbell Collaboration systematic reviews: An assessment of current methods," Campbell Systematic Reviews, John Wiley & Sons, vol. 17(4), December.
    2. Ariel M. Aloe & Ruth Garside, 2021. "Editorial: Types of methods research papers in the journal Campbell Systematic Reviews," Campbell Systematic Reviews, John Wiley & Sons, vol. 17(2), June.
    3. Sanjeev Sharma & Renu Popli & Sajjan Singh & Gunjan Chhabra & Gurpreet Singh Saini & Maninder Singh & Archana Sandhu & Ashutosh Sharma & Rajeev Kumar, 2024. "The Role of 6G Technologies in Advancing Smart City Applications: Opportunities and Challenges," Sustainability, MDPI, vol. 16(16), pages 1-53, August.
    4. Howard White & Bianca Albers & Marie Gaarder & Hege Kornør & Julia Littell & Zack Marshall & Christine Mathew & Terri Pigott & Birte Snilstveit & Hugh Waddington & Vivian Welch, 2020. "Guidance for producing a Campbell evidence and gap map," Campbell Systematic Reviews, John Wiley & Sons, vol. 16(4), December.
    5. Martin Polak & Norbert Tanzer & Per Carlbring, 2022. "PROTOCOL: Effects of virtual reality exposure therapy versus in vivo exposure in treating social anxiety disorder in adults: A systematic review and meta‐analysis," Campbell Systematic Reviews, John Wiley & Sons, vol. 18(3), September.
    6. Chris O'Leary & Rob Ralphs & Jennifer Stevenson & Andrew Smith & Jordan Harrison & Zsolt Kiss & Harry Armitage, 2024. "The effectiveness of abstinence‐based and harm reduction‐based interventions in reducing problematic substance use in adults who are experiencing homelessness in high income countries: A systematic re," Campbell Systematic Reviews, John Wiley & Sons, vol. 20(2), June.
    7. Heather MacDonald & Cozette Comer & Margaret Foster & Patrick R. Labelle & Scott Marsalis & Kate Nyhan & Zahra Premji & Morwenna Rogers & Ryan Splenda & Claire Stansfield & Sarah Young, 2024. "Searching for studies: A guide to information retrieval for Campbell systematic reviews," Campbell Systematic Reviews, John Wiley & Sons, vol. 20(3), September.
    8. Eduardo González‐Cabañes & Trinidad Garcia & Catherine Chase & Jose Carlos Núñez, 2023. "PROTOCOL: Problem solving before instruction (PS‐I) to promote learning and motivation in child and adult students," Campbell Systematic Reviews, John Wiley & Sons, vol. 19(3), September.
    9. Ajay Kumar & Sushil Kumar & Komal & Nirala Ramchiary & Pardeep Singh, 2021. "Role of Traditional Ethnobotanical Knowledge and Indigenous Communities in Achieving Sustainable Development Goals," Sustainability, MDPI, vol. 13(6), pages 1-14, March.
    10. Kylie Burke & Cassandra K. Dittman & Elana J. Forbes & Elizabeth Eggins, 2021. "PROTOCOL: A systematic review and meta‐analysis of randomised controlled trials evaluating the impact of parenting programmes for parents of adolescents (10–18 years) on adolescent mental health outco," Campbell Systematic Reviews, John Wiley & Sons, vol. 17(1), March.
    11. Hugh Waddington & Ada Sonnenfeld & Juliette Finetti & Marie Gaarder & Denny John & Jennifer Stevenson, 2019. "Citizen engagement in public services in low‐ and middle‐income countries: A mixed‐methods systematic review of participation, inclusion, transparency and accountability (PITA) initiatives," Campbell Systematic Reviews, John Wiley & Sons, vol. 15(1-2), June.
    12. Julie Birkenmaier & Brandy Maynard & Hannah Shanks & Harly Blumhagen, 2023. "PROTOCOL: Medical‐financial partnerships for improving financial and health outcomes for lower‐income Americans: A systematic review," Campbell Systematic Reviews, John Wiley & Sons, vol. 19(4), December.
    13. Joseph Lee Betts & Elizabeth Eggins & Ned Chandler‐Mather & Doug Shelton & Haydn Till & Paul Harnett & Sharon Dawe, 2022. "Interventions for improving executive functions in children with foetal alcohol spectrum disorder (FASD): A systematic review," Campbell Systematic Reviews, John Wiley & Sons, vol. 18(4), December.
    14. Hugh Waddington & Jennifer Stevenson & Ada Sonnenfeld & Marie Gaarder, 2018. "PROTOCOL: Participation, inclusion, transparency and accountability (PITA) to improve public services in low‐ and middle‐income countries: a systematic review," Campbell Systematic Reviews, John Wiley & Sons, vol. 14(1), pages 1-69.
    15. Liping Guo & Wenjie Zhou & Xin Xing & Zhipeng Wei & Minyan Yang & Mina Ma & Kehu Yang & Howard White, 2022. "PROTOCOL: Critical appraisal of methodological quality and reporting items of systematic reviews with meta‐analysis in evidence‐based social science in China: A systematic review," Campbell Systematic Reviews, John Wiley & Sons, vol. 18(4), December.
    16. Julia H. Littell & Sarah Young & Therese D. Pigott & M. Antonia Biggs & Trine Munk‐Olsen & Julia R. Steinberg, 2024. "PROTOCOL: Abortion and mental health outcomes: A systematic review and meta‐analysis," Campbell Systematic Reviews, John Wiley & Sons, vol. 20(2), June.
    17. Etienne Lwamba & Shannon Shisler & Will Ridlehoover & Meital Kupfer & Nkululeko Tshabalala & Promise Nduku & Laurenz Langer & Sean Grant & Ada Sonnenfeld & Daniela Anda & John Eyers & Birte Snilstveit, 2022. "Strengthening women's empowerment and gender equality in fragile contexts towards peaceful and inclusive societies: A systematic review and meta‐analysis," Campbell Systematic Reviews, John Wiley & Sons, vol. 18(1), March.
    18. Julie Birkenmaier & Brandy Maynard & Youngmi Kim, 2022. "Interventions designed to improve financial capability: A systematic review," Campbell Systematic Reviews, John Wiley & Sons, vol. 18(1), March.
    19. Emma Reith‐Hall & Paul Montgomery, 2023. "Communication skills training for improving the communicative abilities of student social workers: A systematic review," Campbell Systematic Reviews, John Wiley & Sons, vol. 19(1), March.
    20. Nina T. Dalgaard & Anja Bondebjerg & Rasmus Klokker & Bjørn C. A. Viinholt & Jens Dietrichson, 2022. "Adult/child ratio and group size in early childhood education or care to promote the development of children aged 0–5 years: A systematic review," Campbell Systematic Reviews, John Wiley & Sons, vol. 18(2), June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:camsys:v:20:y:2024:i:3:n:e1432. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://doi.org/10.1111/(ISSN)1891-1803 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.