IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0224135.html
   My bibliography  Save this article

Evaluating risk prediction models for adults with heart failure: A systematic literature review

Author

Listed:
  • Gian Luca Di Tanna
  • Heidi Wirtz
  • Karen L Burrows
  • Gary Globe

Abstract

Background: The ability to predict risk allows healthcare providers to propose which patients might benefit most from certain therapies, and is relevant to payers’ demands to justify clinical and economic value. To understand the robustness of risk prediction models for heart failure (HF), we conducted a systematic literature review to (1) identify HF risk-prediction models, (2) assess statistical approach and extent of validation, (3) identify common variables, and (4) assess risk of bias (ROB). Methods: Literature databases were searched from March 2013 to May 2018 to identify risk prediction models conducted in an out-of-hospital setting in adults with HF. Distinct risk prediction variables were ranked according to outcomes assessed and incorporation into the studies. ROB was assessed using Prediction model Risk Of Bias ASsessment Tool (PROBAST). Results: Of 4720 non-duplicated citations, 40 risk-prediction publications were deemed relevant. Within the 40 publications, 58 models assessed 55 (co)primary outcomes, including all-cause mortality (n = 17), cardiovascular death (n = 9), HF hospitalizations (n = 15), and composite endpoints (n = 14). Few publications reported detail on handling missing data (n = 11; 28%). The discriminatory ability for predicting all-cause mortality, cardiovascular death, and composite endpoints was generally better than for HF hospitalization. 105 distinct predictor variables were identified. Predictors included in >5 publications were: N-terminal prohormone brain-natriuretic peptide, creatinine, blood urea nitrogen, systolic blood pressure, sodium, NYHA class, left ventricular ejection fraction, heart rate, and characteristics including male sex, diabetes, age, and BMI. Only 11/58 (19%) models had overall low ROB, based on our application of PROBAST. In total, 26/58 (45%) models discussed internal validation, and 14/58 (24%) external validation. Conclusions: The majority of the 58 identified risk-prediction models for HF present particular concerns according to ROB assessment, mainly due to lack of validation and calibration. The potential utility of novel approaches such as machine learning tools is yet to be determined. Registration number: The SLR was registered in Prospero (ID: CRD42018100709).

Suggested Citation

  • Gian Luca Di Tanna & Heidi Wirtz & Karen L Burrows & Gary Globe, 2020. "Evaluating risk prediction models for adults with heart failure: A systematic literature review," PLOS ONE, Public Library of Science, vol. 15(1), pages 1-23, January.
  • Handle: RePEc:plo:pone00:0224135
    DOI: 10.1371/journal.pone.0224135
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0224135
    Download Restriction: no

    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0224135&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pone.0224135?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Stephen F Weng & Jenna Reps & Joe Kai & Jonathan M Garibaldi & Nadeem Qureshi, 2017. "Can machine-learning improve cardiovascular risk prediction using routine clinical data?," PLOS ONE, Public Library of Science, vol. 12(4), pages 1-14, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mirza Rizwan Sajid & Bader A. Almehmadi & Waqas Sami & Mansour K. Alzahrani & Noryanti Muhammad & Christophe Chesneau & Asif Hanif & Arshad Ali Khan & Ahmad Shahbaz, 2021. "Development of Nonlaboratory-Based Risk Prediction Models for Cardiovascular Diseases Using Conventional and Machine Learning Approaches," IJERPH, MDPI, vol. 18(23), pages 1-16, November.
    2. Salvatore Tedesco & Martina Andrulli & Markus Åkerlund Larsson & Daniel Kelly & Antti Alamäki & Suzanne Timmons & John Barton & Joan Condell & Brendan O’Flynn & Anna Nordström, 2021. "Comparison of Machine Learning Techniques for Mortality Prediction in a Prospective Cohort of Older Adults," IJERPH, MDPI, vol. 18(23), pages 1-18, December.
    3. Ajay Dev & Sanjay Kumar Malik, 2021. "Artificial Bee Colony Optimized Deep Neural Network Model for Handling Imbalanced Stroke Data: ABC-DNN for Prediction of Stroke," International Journal of E-Health and Medical Communications (IJEHMC), IGI Global, vol. 12(5), pages 67-83, September.
    4. Feihan Lu & Yao Zheng & Harrington Cleveland & Chris Burton & David Madigan, 2018. "Bayesian hierarchical vector autoregressive models for patient-level predictive modeling," PLOS ONE, Public Library of Science, vol. 13(12), pages 1-27, December.
    5. Shinya Suzuki & Takeshi Yamashita & Tsuyoshi Sakama & Takuto Arita & Naoharu Yagi & Takayuki Otsuka & Hiroaki Semba & Hiroto Kano & Shunsuke Matsuno & Yuko Kato & Tokuhisa Uejima & Yuji Oikawa & Minor, 2019. "Comparison of risk models for mortality and cardiovascular events between machine learning and conventional logistic regression analysis," PLOS ONE, Public Library of Science, vol. 14(9), pages 1-14, September.
    6. Ying Wang & Zhicheng Du & Wayne R. Lawrence & Yun Huang & Yu Deng & Yuantao Hao, 2019. "Predicting Hepatitis B Virus Infection Based on Health Examination Data of Community Population," IJERPH, MDPI, vol. 16(23), pages 1-13, December.
    7. Francesco Cappelli & Gianfranco Castronuovo & Salvatore Grimaldi & Vito Telesca, 2024. "Random Forest and Feature Importance Measures for Discriminating the Most Influential Environmental Factors in Predicting Cardiovascular and Respiratory Diseases," IJERPH, MDPI, vol. 21(7), pages 1-21, July.
    8. Shelda Sajeev & Stephanie Champion & Alline Beleigoli & Derek Chew & Richard L. Reed & Dianna J. Magliano & Jonathan E. Shaw & Roger L. Milne & Sarah Appleton & Tiffany K. Gill & Anthony Maeder, 2021. "Predicting Australian Adults at High Risk of Cardiovascular Disease Mortality Using Standard Risk Factors and Machine Learning," IJERPH, MDPI, vol. 18(6), pages 1-14, March.
    9. Emily J MacKay & Michael D Stubna & Corey Chivers & Michael E Draugelis & William J Hanson & Nimesh D Desai & Peter W Groeneveld, 2021. "Application of machine learning approaches to administrative claims data to predict clinical outcomes in medical and surgical patient populations," PLOS ONE, Public Library of Science, vol. 16(6), pages 1-14, June.
    10. Woo Suk Hong & Adrian Daniel Haimovich & R Andrew Taylor, 2018. "Predicting hospital admission at emergency department triage using machine learning," PLOS ONE, Public Library of Science, vol. 13(7), pages 1-13, July.
    11. Adrian Richter & Julia Truthmann & Jean-François Chenot & Carsten Oliver Schmidt, 2021. "Predicting Physician Consultations for Low Back Pain Using Claims Data and Population-Based Cohort Data—An Interpretable Machine Learning Approach," IJERPH, MDPI, vol. 18(22), pages 1-14, November.
    12. Rafał Niemiec & Irmina Morawska & Maria Stec & Wiktoria Kuczmik & Andrzej S. Swinarew & Arkadiusz Stanula & Katarzyna Mizia-Stec, 2022. "ARNI in HFrEF—One-Centre Experience in the Era before the 2021 ESC HF Recommendations," IJERPH, MDPI, vol. 19(4), pages 1-12, February.
    13. Dohyun Kim & Sungmin You & Soonwon So & Jongshill Lee & Sunhyun Yook & Dong Pyo Jang & In Young Kim & Eunkyoung Park & Kyeongwon Cho & Won Chul Cha & Dong Wook Shin & Baek Hwan Cho & Hoon-Ki Park, 2018. "A data-driven artificial intelligence model for remote triage in the prehospital environment," PLOS ONE, Public Library of Science, vol. 13(10), pages 1-14, October.
    14. Hoa Thi Nguyen & Claudia M. Denkinger & Stephan Brenner & Lisa Koeppel & Lucia Brugnara & Robin Burk & Michael Knop & Till Bärnighausen & Andreas Deckert & Manuela De Allegri, 2023. "Cost and cost-effectiveness of four different SARS-CoV-2 active surveillance strategies: evidence from a randomised control trial in Germany," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 24(9), pages 1545-1559, December.
    15. Sharan Srinivas, 2020. "A Machine Learning-Based Approach for Predicting Patient Punctuality in Ambulatory Care Centers," IJERPH, MDPI, vol. 17(10), pages 1-15, May.
    16. Syed Waseem Abbas Sherazi & Jang-Whan Bae & Jong Yun Lee, 2021. "A soft voting ensemble classifier for early prediction and diagnosis of occurrences of major adverse cardiovascular events for STEMI and NSTEMI during 2-year follow-up in patients with acute coronary ," PLOS ONE, Public Library of Science, vol. 16(6), pages 1-20, June.
    17. Stephen F Weng & Luis Vaz & Nadeem Qureshi & Joe Kai, 2019. "Prediction of premature all-cause mortality: A prospective general population cohort study comparing machine-learning and standard epidemiological approaches," PLOS ONE, Public Library of Science, vol. 14(3), pages 1-22, March.
    18. Alexander Engels & Katrin C Reber & Ivonne Lindlbauer & Kilian Rapp & Gisela Büchele & Jochen Klenk & Andreas Meid & Clemens Becker & Hans-Helmut König, 2020. "Osteoporotic hip fracture prediction from risk factors available in administrative claims data – A machine learning approach," PLOS ONE, Public Library of Science, vol. 15(5), pages 1-14, May.
    19. Cynthia Rudin & Berk Ustun, 2018. "Optimized Scoring Systems: Toward Trust in Machine Learning for Healthcare and Criminal Justice," Interfaces, INFORMS, vol. 48(5), pages 449-466, October.
    20. Pablo Gonzalez Ginestet & Ales Kotalik & David M. Vock & Julian Wolfson & Erin E. Gabriel, 2021. "Stacked inverse probability of censoring weighted bagging: A case study in the InfCareHIV Register," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 70(1), pages 51-65, January.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0224135. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.