IDEAS home Printed from https://ideas.repec.org/a/spr/stabio/v15y2023i2d10.1007_s12561-023-09362-0.html
   My bibliography  Save this article

Evaluating Dynamic Discrimination Performance of Risk Prediction Models for Survival Outcomes

Author

Listed:
  • Jing Zhang

    (The University of Texas Health Science Center at Houston)

  • Jing Ning

    (The University of Texas MD Anderson Cancer Center)

  • Ruosha Li

    (The University of Texas Health Science Center at Houston)

Abstract

Risk prediction models for survival outcomes are widely applied in medical research to predict future risk for the occurrence of the event. In many clinical studies, the biomarker data are measured repeatedly over time. To facilitate timely disease prognosis and decision making, many dynamic prediction models have been developed and generate predictions on a real-time basis. As a dynamic prediction model updates an individual’s risk prediction over time based on new measurements, it is often important to examine how well the model performs at different measurement times and prediction times. In this article, we propose a two-dimensional area under curve (AUC) measure for dynamic prediction models and develop associated estimation and inference procedures. The estimation procedures are discussed under two types of biomarker measurement schedules: regular visits and irregular visits. The model parameters are estimated effectively by maximizing a pseudo-partial likelihood function. We apply the proposed method to a renal transplantation study to evaluate the discrimination performance of dynamic prediction models based on longitudinal biomarkers for graft failure.

Suggested Citation

  • Jing Zhang & Jing Ning & Ruosha Li, 2023. "Evaluating Dynamic Discrimination Performance of Risk Prediction Models for Survival Outcomes," Statistics in Biosciences, Springer;International Chinese Statistical Association, vol. 15(2), pages 353-371, July.
  • Handle: RePEc:spr:stabio:v:15:y:2023:i:2:d:10.1007_s12561-023-09362-0
    DOI: 10.1007/s12561-023-09362-0
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s12561-023-09362-0
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s12561-023-09362-0?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Yingye Zheng & Patrick J. Heagerty, 2007. "Prospective Accuracy for Longitudinal Markers," Biometrics, The International Biometric Society, vol. 63(2), pages 332-341, June.
    2. Ruosha Li & Jing Ning & Ziding Feng, 2022. "Estimation and inference of predictive discrimination for survival outcome risk prediction models," Lifetime Data Analysis: An International Journal Devoted to Statistical Methods and Applications for Time-to-Event Data, Springer, vol. 28(2), pages 219-240, April.
    3. P. Saha & P. J. Heagerty, 2010. "Time-Dependent Predictive Accuracy in the Presence of Competing Risks," Biometrics, The International Biometric Society, vol. 66(4), pages 999-1011, December.
    4. Liang Li & Sheng Luo & Bo Hu & Tom Greene, 2017. "Dynamic Prediction of Renal Failure Using Longitudinal Biomarkers in a Cohort Study of Chronic Kidney Disease," Statistics in Biosciences, Springer;International Chinese Statistical Association, vol. 9(2), pages 357-378, December.
    5. Yingye Zheng & Patrick Heagerty, 2004. "Semiparametric Estimation of Time-Dependent: ROC Curves for Longitudinal Marker Data," UW Biostatistics Working Paper Series 1052, Berkeley Electronic Press.
    6. Patrick J. Heagerty & Yingye Zheng, 2005. "Survival Model Predictive Accuracy and ROC Curves," Biometrics, The International Biometric Society, vol. 61(1), pages 92-105, March.
    7. R. Schoop & E. Graf & M. Schumacher, 2008. "Quantifying the Predictive Performance of Prognostic Models for Censored Survival Data with Time-Dependent Covariates," Biometrics, The International Biometric Society, vol. 64(2), pages 603-610, June.
    8. Hans C. Van Houwelingen, 2007. "Dynamic Prediction by Landmarking in Event History Analysis," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 34(1), pages 70-85, March.
    9. Yingye Zheng & Patrick J. Heagerty, 2005. "Partly Conditional Survival Models for Longitudinal Data," Biometrics, The International Biometric Society, vol. 61(2), pages 379-391, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Marlena Maziarz & Patrick Heagerty & Tianxi Cai & Yingye Zheng, 2017. "On longitudinal prediction with time-to-event outcome: Comparison of modeling options," Biometrics, The International Biometric Society, vol. 73(1), pages 83-93, March.
    2. Liang Li & Sheng Luo & Bo Hu & Tom Greene, 2017. "Dynamic Prediction of Renal Failure Using Longitudinal Biomarkers in a Cohort Study of Chronic Kidney Disease," Statistics in Biosciences, Springer;International Chinese Statistical Association, vol. 9(2), pages 357-378, December.
    3. P. Saha & P. J. Heagerty, 2010. "Time-Dependent Predictive Accuracy in the Presence of Competing Risks," Biometrics, The International Biometric Society, vol. 66(4), pages 999-1011, December.
    4. Yingye Zheng & Patrick J. Heagerty, 2007. "Prospective Accuracy for Longitudinal Markers," Biometrics, The International Biometric Society, vol. 63(2), pages 332-341, June.
    5. Yingye Zheng & Tianxi Cai & Yuying Jin & Ziding Feng, 2012. "Evaluating Prognostic Accuracy of Biomarkers under Competing Risk," Biometrics, The International Biometric Society, vol. 68(2), pages 388-396, June.
    6. Daniel Commenges & Benoit Liquet & Cécile Proust-Lima, 2012. "Choice of Prognostic Estimators in Joint Models by Estimating Differences of Expected Conditional Kullback–Leibler Risks," Biometrics, The International Biometric Society, vol. 68(2), pages 380-387, June.
    7. R. Schoop & E. Graf & M. Schumacher, 2008. "Quantifying the Predictive Performance of Prognostic Models for Censored Survival Data with Time-Dependent Covariates," Biometrics, The International Biometric Society, vol. 64(2), pages 603-610, June.
    8. Giulia Barbati & Alessio Farcomeni, 2018. "Prognostic assessment of repeatedly measured time-dependent biomarkers, with application to dilated cardiomyopathy," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 27(3), pages 545-557, August.
    9. Pablo Mart�nez-Camblor & Jacobo de U�a-�lvarez & Carmen D�az Corte, 2015. "Expanded renal transplantation: a competing risk model approach," Journal of Applied Statistics, Taylor & Francis Journals, vol. 42(12), pages 2539-2553, December.
    10. Dimitris Rizopoulos, 2011. "Dynamic Predictions and Prospective Accuracy in Joint Models for Longitudinal and Time-to-Event Data," Biometrics, The International Biometric Society, vol. 67(3), pages 819-829, September.
    11. Weining Shen & Jing Ning & Ying Yuan, 2015. "A direct method to evaluate the time-dependent predictive accuracy for biomarkers," Biometrics, The International Biometric Society, vol. 71(2), pages 439-449, June.
    12. Matthias Schmid & Thomas Hielscher & Thomas Augustin & Olaf Gefeller, 2011. "A Robust Alternative to the Schemper–Henderson Estimator of Prediction Error," Biometrics, The International Biometric Society, vol. 67(2), pages 524-535, June.
    13. Shanshan Li & Yang Ning, 2015. "Estimation of covariate‐specific time‐dependent ROC curves in the presence of missing biomarkers," Biometrics, The International Biometric Society, vol. 71(3), pages 666-676, September.
    14. Dimitris Rizopoulos & Laura A. Hatfield & Bradley P. Carlin & Johanna J. M. Takkenberg, 2014. "Combining Dynamic Predictions From Joint Models for Longitudinal and Time-to-Event Data Using Bayesian Model Averaging," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(508), pages 1385-1397, December.
    15. Zijing Yang & Chengfeng Zhang & Yawen Hou & Zheng Chen, 2023. "Analysis of dynamic restricted mean survival time based on pseudo‐observations," Biometrics, The International Biometric Society, vol. 79(4), pages 3690-3700, December.
    16. C. Jason Liang & Patrick J. Heagerty, 2017. "Rejoinder to discussions on: A risk-based measure of time-varying prognostic discrimination for survival models," Biometrics, The International Biometric Society, vol. 73(3), pages 745-748, September.
    17. van Geloven, N. & He, Y. & Zwinderman, A.H. & Putter, H., 2021. "Estimation of incident dynamic AUC in practice," Computational Statistics & Data Analysis, Elsevier, vol. 154(C).
    18. Sean M. Devlin & Mithat Gönen & Glenn Heller, 2020. "Measuring the temporal prognostic utility of a baseline risk score," Lifetime Data Analysis: An International Journal Devoted to Statistical Methods and Applications for Time-to-Event Data, Springer, vol. 26(4), pages 856-871, October.
    19. Qi Gong & Douglas E. Schaubel, 2013. "Partly Conditional Estimation of the Effect of a Time-Dependent Factor in the Presence of Dependent Censoring," Biometrics, The International Biometric Society, vol. 69(2), pages 338-347, June.
    20. Aasthaa Bansal & Patrick J. Heagerty, 2018. "A Tutorial on Evaluating the Time-Varying Discrimination Accuracy of Survival Models Used in Dynamic Decision Making," Medical Decision Making, , vol. 38(8), pages 904-916, November.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:stabio:v:15:y:2023:i:2:d:10.1007_s12561-023-09362-0. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.