IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0086277.html
   My bibliography  Save this article

Feature Engineering and a Proposed Decision-Support System for Systematic Reviewers of Medical Evidence

Author

Listed:
  • Tanja Bekhuis
  • Eugene Tseytlin
  • Kevin J Mitchell
  • Dina Demner-Fushman

Abstract

Objectives: Evidence-based medicine depends on the timely synthesis of research findings. An important source of synthesized evidence resides in systematic reviews. However, a bottleneck in review production involves dual screening of citations with titles and abstracts to find eligible studies. For this research, we tested the effect of various kinds of textual information (features) on performance of a machine learning classifier. Based on our findings, we propose an automated system to reduce screeing burden, as well as offer quality assurance. Methods: We built a database of citations from 5 systematic reviews that varied with respect to domain, topic, and sponsor. Consensus judgments regarding eligibility were inferred from published reports. We extracted 5 feature sets from citations: alphabetic, alphanumeric+, indexing, features mapped to concepts in systematic reviews, and topic models. To simulate a two-person team, we divided the data into random halves. We optimized the parameters of a Bayesian classifier, then trained and tested models on alternate data halves. Overall, we conducted 50 independent tests. Results: All tests of summary performance (mean F3) surpassed the corresponding baseline, P

Suggested Citation

  • Tanja Bekhuis & Eugene Tseytlin & Kevin J Mitchell & Dina Demner-Fushman, 2014. "Feature Engineering and a Proposed Decision-Support System for Systematic Reviewers of Medical Evidence," PLOS ONE, Public Library of Science, vol. 9(1), pages 1-10, January.
    • Handle: RePEc:plo:pone00:0086277
    DOI: 10.1371/journal.pone.0086277
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0086277
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0086277&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0086277?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Siddhartha R. Dalal & Paul G. Shekelle & Susanne Hempel & Sydne J. Newberry & Aneesa Motala & Kanaka D. Shetty, 2013. "A Pilot Study Using Machine Learning and Domain Knowledge to Facilitate Comparative Effectiveness Review Updating," Medical Decision Making, , vol. 33(3), pages 343-355, April.
    2. Joanne Yaffe & Paul Montgomery & Sally Hopewell & Lindsay Dianne Shepard, 2012. "Empty Reviews: A Description and Consideration of Cochrane Systematic Reviews with No Included Studies," PLOS ONE, Public Library of Science, vol. 7(5), pages 1-7, May.
    3. Hilda Bastian & Paul Glasziou & Iain Chalmers, 2010. "Seventy-Five Trials and Eleven Systematic Reviews a Day: How Will We Ever Keep Up?," PLOS Medicine, Public Library of Science, vol. 7(9), pages 1-6, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dreesens, Dunja & Kremer, Leontien & Burgers, Jako & van der Weijden, Trudy, 2020. "Lost in definitions: Reducing duplication and clarifying definitions of knowledge and decision support tools. A RAND-modified Delphi consensus study," Health Policy, Elsevier, vol. 124(5), pages 531-539.
    2. Blake M. Louscher & Veerasathpurush Allareddy & Satheesh Elangovan, 2019. "Predictors of Citations of Systematic Reviews in Oral Implantology: A Cross-Sectional Bibliometric Analysis," SAGE Open, , vol. 9(1), pages 21582440198, March.
    3. Hansen, Henrik & Trifkovic, Neda, 2013. "Systematic Reviews: Questions, Methods and Usage," MPRA Paper 47993, University Library of Munich, Germany.
    4. Mark J Bolland & Andrew Grey, 2014. "A Case Study of Discordant Overlapping Meta-Analyses: Vitamin D Supplements and Fracture," PLOS ONE, Public Library of Science, vol. 9(12), pages 1-20, December.
    5. Shi, Xuanyu & Du, Jian, 2022. "Distinguishing transformative from incremental clinical evidence: A classifier of clinical research using textual features from abstracts and citing sentences," Journal of Informetrics, Elsevier, vol. 16(2).
    6. Diana J. Arango & Matthew Morton & Floriza Gennari & Sveinung Kiplesund & Mary Ellsberg, 2014. "Interventions to Prevent or Reduce Violence Against Women and Girls : A Systematic Review of Reviews," World Bank Publications - Reports 21035, The World Bank Group.
    7. Jason Portenoy & Jevin D. West, 2020. "Constructing and evaluating automated literature review systems," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 3233-3251, December.
    8. Dreesens, Dunja & Kremer, Leontien & van der Weijden, Trudy, 2019. "The Dutch chaos case: A scoping review of knowledge and decision support tools available to clinicians in the Netherlands," Health Policy, Elsevier, vol. 123(12), pages 1288-1297.
    9. Joerg J Meerpohl & Florian Herrle & Gerd Antes & Erik von Elm, 2012. "Scientific Value of Systematic Reviews: Survey of Editors of Core Clinical Journals," PLOS ONE, Public Library of Science, vol. 7(5), pages 1-5, May.
    10. Lucy Turner & James Galipeau & Chantelle Garritty & Eric Manheimer & L Susan Wieland & Fatemeh Yazdi & David Moher, 2013. "An Evaluation of Epidemiological and Reporting Characteristics of Complementary and Alternative Medicine (CAM) Systematic Reviews (SRs)," PLOS ONE, Public Library of Science, vol. 8(1), pages 1-11, January.
    11. K. M. Saif‐Ur‐Rahman & Md. Hasan & Shahed Hossain & Iqbal Anwar & Yoshihisa Hirakawa & Hiroshi Yatsuya, 2022. "Prioritization and sequential exclusion of articles in systematic reviews," Campbell Systematic Reviews, John Wiley & Sons, vol. 18(2), June.
    12. Monique Heijmans & Rune Poortvliet & Marieke Van der Gaag & Ana I. González-González & Jessica Beltran Puerta & Carlos Canelo-Aybar & Claudia Valli & Marta Ballester & Claudio Rocha & Montserrat León , 2022. "Using a Taxonomy to Systematically Identify and Describe Self-Management Interventions Components in Randomized Trials for COPD," IJERPH, MDPI, vol. 19(19), pages 1-24, October.
    13. Tanja Burgard & Holger Steinmetz, 2023. "Evidence in management science related to psychology: benefits, tools, and an example of a community-augmented meta-analysis," Management Review Quarterly, Springer, vol. 73(3), pages 1135-1150, September.
    14. Jorden A Cummings & Jessica M Zagrodney & T Eugene Day, 2015. "Impact of Open Data Policies on Consent to Participate in Human Subjects Research: Discrepancies between Participant Action and Reported Concerns," PLOS ONE, Public Library of Science, vol. 10(5), pages 1-11, May.
    15. Peter J Gill & Kay Yee Wang & David Mant & Lisa Hartling & Carl Heneghan & Rafael Perera & Terry Klassen & Anthony Harnden, 2011. "The Evidence Base for Interventions Delivered to Children in Primary Care: An Overview of Cochrane Systematic Reviews," PLOS ONE, Public Library of Science, vol. 6(8), pages 1-9, August.
    16. Laura Sheble, 2017. "Macro‐level diffusion of a methodological knowledge innovation: Research synthesis methods, 1972–2011," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 68(12), pages 2693-2708, December.
    17. Md Mahbub Hossain & Abida Sultana & Samia Tasnim & Qiping Fan & Ping Ma & E Lisako J McKyer & Neetu Purohit, 2020. "Prevalence of mental disorders among people who are homeless: An umbrella review," International Journal of Social Psychiatry, , vol. 66(6), pages 528-541, September.
    18. Costa, King & Ntsobi, Mfanelo Patrick, 2023. "A Systematic Review of the Potential for Promotion of Southern Epistemologies in Educational Research: Ubuntu Philosophy as a Research Paradigm, a Conceptual Model," AfricArxiv qshp8, Center for Open Science.
    19. Means, Stephanie N. & Magura, Stephen & Burkhardt, Jason T. & Schröter, Daniela C. & Coryn, Chris L.S., 2015. "Comparing rating paradigms for evidence-based program registers in behavioral health: Evidentiary criteria and implications for assessing programs," Evaluation and Program Planning, Elsevier, vol. 48(C), pages 100-116.
    20. Porjai Pattanittum & Malinee Laopaiboon & David Moher & Pisake Lumbiganon & Chetta Ngamjarus, 2012. "A Comparison of Statistical Methods for Identifying Out-of-Date Systematic Reviews," PLOS ONE, Public Library of Science, vol. 7(11), pages 1-6, November.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0086277. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.