IDEAS home Printed from https://ideas.repec.org/a/sae/medema/v38y2018i8p904-916.html
   My bibliography  Save this article

A Tutorial on Evaluating the Time-Varying Discrimination Accuracy of Survival Models Used in Dynamic Decision Making

Author

Listed:
  • Aasthaa Bansal
  • Patrick J. Heagerty

Abstract

Many medical decisions involve the use of dynamic information collected on individual patients toward predicting likely transitions in their future health status. If accurate predictions are developed, then a prognostic model can identify patients at greatest risk for future adverse events and may be used clinically to define populations appropriate for targeted intervention. In practice, a prognostic model is often used to guide decisions at multiple time points over the course of disease, and classification performance (i.e., sensitivity and specificity) for distinguishing high-risk v. low-risk individuals may vary over time as an individual’s disease status and prognostic information change. In this tutorial, we detail contemporary statistical methods that can characterize the time-varying accuracy of prognostic survival models when used for dynamic decision making. Although statistical methods for evaluating prognostic models with simple binary outcomes are well established, methods appropriate for survival outcomes are less well known and require time-dependent extensions of sensitivity and specificity to fully characterize longitudinal biomarkers or models. The methods we review are particularly important in that they allow for appropriate handling of censored outcomes commonly encountered with event time data. We highlight the importance of determining whether clinical interest is in predicting cumulative (or prevalent) cases over a fixed future time interval v. predicting incident cases over a range of follow-up times and whether patient information is static or updated over time. We discuss implementation of time-dependent receiver operating characteristic approaches using relevant R statistical software packages. The statistical summaries are illustrated using a liver prognostic model to guide transplantation in primary biliary cirrhosis.

Suggested Citation

  • Aasthaa Bansal & Patrick J. Heagerty, 2018. "A Tutorial on Evaluating the Time-Varying Discrimination Accuracy of Survival Models Used in Dynamic Decision Making," Medical Decision Making, , vol. 38(8), pages 904-916, November.
  • Handle: RePEc:sae:medema:v:38:y:2018:i:8:p:904-916
    DOI: 10.1177/0272989X18801312
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0272989X18801312
    Download Restriction: no

    File URL: https://libkey.io/10.1177/0272989X18801312?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Yingye Zheng & Patrick J. Heagerty, 2007. "Prospective Accuracy for Longitudinal Markers," Biometrics, The International Biometric Society, vol. 63(2), pages 332-341, June.
    2. Patrick J. Heagerty & Yingye Zheng, 2005. "Survival Model Predictive Accuracy and ROC Curves," Biometrics, The International Biometric Society, vol. 61(1), pages 92-105, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ayis Pyrros & Stephen M. Borstelmann & Ramana Mantravadi & Zachary Zaiman & Kaesha Thomas & Brandon Price & Eugene Greenstein & Nasir Siddiqui & Melinda Willis & Ihar Shulhan & John Hines-Shah & Jeann, 2023. "Opportunistic detection of type 2 diabetes using deep learning from frontal chest radiographs," Nature Communications, Nature, vol. 14(1), pages 1-12, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. C. Jason Liang & Patrick J. Heagerty, 2017. "Rejoinder to discussions on: A risk-based measure of time-varying prognostic discrimination for survival models," Biometrics, The International Biometric Society, vol. 73(3), pages 745-748, September.
    2. van Geloven, N. & He, Y. & Zwinderman, A.H. & Putter, H., 2021. "Estimation of incident dynamic AUC in practice," Computational Statistics & Data Analysis, Elsevier, vol. 154(C).
    3. Dimitris Rizopoulos, 2011. "Dynamic Predictions and Prospective Accuracy in Joint Models for Longitudinal and Time-to-Event Data," Biometrics, The International Biometric Society, vol. 67(3), pages 819-829, September.
    4. P. Saha & P. J. Heagerty, 2010. "Time-Dependent Predictive Accuracy in the Presence of Competing Risks," Biometrics, The International Biometric Society, vol. 66(4), pages 999-1011, December.
    5. Jing Zhang & Jing Ning & Ruosha Li, 2023. "Evaluating Dynamic Discrimination Performance of Risk Prediction Models for Survival Outcomes," Statistics in Biosciences, Springer;International Chinese Statistical Association, vol. 15(2), pages 353-371, July.
    6. Liang Li & Sheng Luo & Bo Hu & Tom Greene, 2017. "Dynamic Prediction of Renal Failure Using Longitudinal Biomarkers in a Cohort Study of Chronic Kidney Disease," Statistics in Biosciences, Springer;International Chinese Statistical Association, vol. 9(2), pages 357-378, December.
    7. Yingye Zheng & Tianxi Cai & Yuying Jin & Ziding Feng, 2012. "Evaluating Prognostic Accuracy of Biomarkers under Competing Risk," Biometrics, The International Biometric Society, vol. 68(2), pages 388-396, June.
    8. Robin Van Oirbeek & Emmanuel Lesaffre, 2018. "An Investigation of the Discriminatory Ability of the Clustering Effect of the Frailty Survival Model," Biostatistics and Biometrics Open Access Journal, Juniper Publishers Inc., vol. 6(3), pages 87-98, April.
    9. Yuanjia Wang & Huaihou Chen & Runze Li & Naihua Duan & Roberto Lewis-Fernández, 2011. "Prediction-Based Structured Variable Selection through the Receiver Operating Characteristic Curves," Biometrics, The International Biometric Society, vol. 67(3), pages 896-905, September.
    10. Kevin He & Yue Wang & Xiang Zhou & Han Xu & Can Huang, 2019. "An improved variable selection procedure for adaptive Lasso in high-dimensional survival analysis," Lifetime Data Analysis: An International Journal Devoted to Statistical Methods and Applications for Time-to-Event Data, Springer, vol. 25(3), pages 569-585, July.
    11. Weining Shen & Jing Ning & Ying Yuan, 2015. "A direct method to evaluate the time-dependent predictive accuracy for biomarkers," Biometrics, The International Biometric Society, vol. 71(2), pages 439-449, June.
    12. Shanshan Li & Yang Ning, 2015. "Estimation of covariate‐specific time‐dependent ROC curves in the presence of missing biomarkers," Biometrics, The International Biometric Society, vol. 71(3), pages 666-676, September.
    13. Hannes Kröger & Rasmus Hoffmann, 2018. "The association between CVD-related biomarkers and mortality in the Health and Retirement Survey," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 38(62), pages 1933-2002.
    14. Yingye Zheng & Tianxi Cai & Ziding Feng, 2006. "Application of the Time-Dependent ROC Curves for Prognostic Accuracy with Multiple Biomarkers," Biometrics, The International Biometric Society, vol. 62(1), pages 279-287, March.
    15. C. Jason Liang & Patrick J. Heagerty, 2017. "A risk-based measure of time-varying prognostic discrimination for survival models," Biometrics, The International Biometric Society, vol. 73(3), pages 725-734, September.
    16. Baccini, Leonardo & Koenig-Archibugi, Mathias, 2014. "Why do states commit to international labor standards?: interdependent ratification of core ILO conventions, 1948-2009," LSE Research Online Documents on Economics 57665, London School of Economics and Political Science, LSE Library.
    17. Yingye Zheng & Patrick J. Heagerty, 2007. "Prospective Accuracy for Longitudinal Markers," Biometrics, The International Biometric Society, vol. 63(2), pages 332-341, June.
    18. Yoon, Dahlnym & Eher, Reinhard & Mokros, Andreas, 2022. "Incremental validity of the Psychopathy Checklist-Revised above and beyond the diagnosis of antisocial personality disorder regarding recidivism in sexual offenders," Journal of Criminal Justice, Elsevier, vol. 80(C).
    19. Kröger, Hannes & Hoffmann, Rasmus, 2018. "The Association between Cvd-Related Biomarkers and Mortality in the Health and Retirement Survey," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 38, pages 1933-2002.
    20. Mei-Cheng Wang & Shanshan Li, 2012. "Bivariate Marker Measurements and ROC Analysis," Biometrics, The International Biometric Society, vol. 68(4), pages 1207-1218, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:medema:v:38:y:2018:i:8:p:904-916. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.