IDEAS home Printed from https://ideas.repec.org/a/spr/drugsa/v42y2019i9d10.1007_s40264-019-00831-4.html
   My bibliography  Save this article

Enabling Data-Driven Clinical Quality Assurance: Predicting Adverse Event Reporting in Clinical Trials Using Machine Learning

Author

Listed:
  • Timothé Ménard

    (F. Hoffmann-La Roche)

  • Yves Barmaz

    (F. Hoffmann-La Roche)

  • Björn Koneswarakantha

    (F. Hoffmann-La Roche)

  • Rich Bowling

    (Genentech - A Member of the Roche group)

  • Leszek Popko

    (F. Hoffmann-La Roche)

Abstract

Introduction Adverse event (AE) under-reporting has been a recurrent issue raised during health authorities Good Clinical Practices (GCP) inspections and audits. Moreover, safety under-reporting poses a risk to patient safety and data integrity. The current clinical Quality Assurance (QA) practices used to detect AE under-reporting rely heavily on investigator site and study audits. Yet several sponsors and institutions have had repeated findings related to safety reporting, and this has led to delays in regulatory submissions. Recent developments in data management and IT systems allow data scientists to apply techniques such as machine learning to detect AE under-reporting in an automated fashion. Objective In this project, we developed a predictive model that enables Roche/Genentech Quality Program Leads oversight of AE reporting at the program, study, site, and patient level. This project was part of a broader effort at Roche/Genentech Product Development Quality to apply advanced analytics to augment and complement traditional clinical QA approaches. Method We used a curated data set from 104 completed Roche/Genentech sponsored clinical studies to train a machine learning model to predict the expected number of AEs. Our final model used 54 features built on patient (e.g., demographics, vitals) and study attributes (e.g., molecule class, disease area). Results In order to evaluate model performance, we tested how well it would detect simulated test cases based on data not used for model training. For relevant simulation scenarios of 25%, 50%, and 75% under-reporting on the site level, our model scored an area under the curve (AUC) of the receiver operating characteristic (ROC) curve of 0.62, 0.79, and 0.92, respectively. Conclusion The model has been deployed to evaluate safety reporting performance in a set of ongoing studies in the form of a QA/dashboard cockpit available to Roche Quality Program Leads. Applicability and production performance will be assessed over the next 12–24 months in which we will develop a validation strategy to fully integrate our model into Roche QA processes.

Suggested Citation

  • Timothé Ménard & Yves Barmaz & Björn Koneswarakantha & Rich Bowling & Leszek Popko, 2019. "Enabling Data-Driven Clinical Quality Assurance: Predicting Adverse Event Reporting in Clinical Trials Using Machine Learning," Drug Safety, Springer, vol. 42(9), pages 1045-1053, September.
  • Handle: RePEc:spr:drugsa:v:42:y:2019:i:9:d:10.1007_s40264-019-00831-4
    DOI: 10.1007/s40264-019-00831-4
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s40264-019-00831-4
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s40264-019-00831-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Timothé Ménard & Björn Koneswarakantha & Donato Rolo & Yves Barmaz & Leszek Popko & Rich Bowling, 2020. "Follow-Up on the Use of Machine Learning in Clinical Quality Assurance: Can We Detect Adverse Event Under-Reporting in Oncology Trials?," Drug Safety, Springer, vol. 43(3), pages 295-296, March.
    2. Yves Barmaz & Timothé Ménard, 2021. "Bayesian Modeling for the Detection of Adverse Events Underreporting in Clinical Trials," Drug Safety, Springer, vol. 44(9), pages 949-955, September.
    3. Björn Koneswarakantha & Yves Barmaz & Timothé Ménard & Donato Rolo, 2021. "Follow-up on the Use of Advanced Analytics for Clinical Quality Assurance: Bootstrap Resampling to Enhance Detection of Adverse Event Under-Reporting," Drug Safety, Springer, vol. 44(1), pages 121-123, January.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:drugsa:v:42:y:2019:i:9:d:10.1007_s40264-019-00831-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com/economics/journal/40264 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.