IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0221433.html
   My bibliography  Save this article

Cumulative ROC curves for discriminating three or more ordinal outcomes with cutpoints on a shared continuous measurement scale

Author

Listed:
  • B Rey deCastro

Abstract

Cumulative receiver operator characteristic (ROC) curve analysis extends classic ROC curve analysis to discriminate three or more ordinal outcome levels on a shared continuous scale. The procedure combines cumulative logit regression with a cumulative extension to the ROC curve and performs as expected with ternary (three-level) ordinal outcomes under a variety of simulated conditions (unbalanced data, proportional and non-proportional odds, areas under the ROC curve [AUCs] from 0.70 to 0.95). Simulations also compared several criteria for selecting cutpoints to discriminate outcome levels: the Youden Index, Matthews Correlation Coefficient, Total Accuracy, and Markedness. Total Accuracy demonstrated the least absolute percent-bias. Cutpoints computed from maximum likelihood regression parameters demonstrated bias that was often negligible. The procedure was also applied to publicly available data related to computer imaging and biomarker exposure science, yielding good to excellent AUCs, as well as cutpoints with sensitivities and specificities of commensurate quality. Implementation of cumulative ROC curve analysis and extension to more than three outcome levels are straightforward. The author’s programs for ternary ordinal outcomes are publicly available.

Suggested Citation

  • B Rey deCastro, 2019. "Cumulative ROC curves for discriminating three or more ordinal outcomes with cutpoints on a shared continuous measurement scale," PLOS ONE, Public Library of Science, vol. 14(8), pages 1-16, August.
    • Handle: RePEc:plo:pone00:0221433
    DOI: 10.1371/journal.pone.0221433
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0221433
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0221433&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0221433?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Anna N. Angelos Tosteson & Colin B. Begg, 1988. "A General Regression Methodology for ROC Curve Estimation," Medical Decision Making, , vol. 8(3), pages 204-215, August.
    2. Hirschberg, Joe & Lye, Jenny, 2010. "A Geometric Comparison of the Delta and Fieller Confidence Intervals," The American Statistician, American Statistical Association, vol. 64(3), pages 234-241.
    3. Jing Qin, 2003. "Using logistic regression procedures for estimating receiver operating characteristic curves," Biometrika, Biometrika Trust, vol. 90(3), pages 585-596, September.
    4. Jan Budczies & Frederick Klauschen & Bruno V Sinn & Balázs Győrffy & Wolfgang D Schmitt & Silvia Darb-Esfahani & Carsten Denkert, 2012. "Cutoff Finder: A Comprehensive and Straightforward Web Application Enabling Rapid Biomarker Cutoff Optimization," PLOS ONE, Public Library of Science, vol. 7(12), pages 1-7, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Joe Hirschberg & Jenny Lye, 2017. "Alternative Graphical Representations of the Confidence Intervals for the Structural Coefficient from Exactly Identified Two-Stage Least Squares," Department of Economics - Working Papers Series 2026, The University of Melbourne.
    2. Ziyi Li & Yijian Huang & Dattatraya Patil & Martin G. Sanda, 2023. "Covariate adjustment in continuous biomarker assessment," Biometrics, The International Biometric Society, vol. 79(1), pages 39-48, March.
    3. Hirschberg, Joe & Lye, Jenny, 2017. "Inverting the indirect—The ellipse and the boomerang: Visualizing the confidence intervals of the structural coefficient from two-stage least squares," Journal of Econometrics, Elsevier, vol. 199(2), pages 173-183.
    4. Cheam, Amay S.M. & McNicholas, Paul D., 2016. "Modelling receiver operating characteristic curves using Gaussian mixtures," Computational Statistics & Data Analysis, Elsevier, vol. 93(C), pages 192-208.
    5. Nasim Vahabi & Anoshirvan Kazemnejad & Somnath Datta, 2018. "A Marginalized Overdispersed Location Scale Model for Clustered Ordinal Data," Sankhya B: The Indian Journal of Statistics, Springer;Indian Statistical Institute, vol. 80(1), pages 103-134, December.
    6. Jean-Marie Dufour & Emmanuel Flachaire & Lynda Khalaf & Abdallah Zalghout, 2020. "Identification-Robust Inequality Analysis," Cahiers de recherche 03-2020, Centre interuniversitaire de recherche en économie quantitative, CIREQ.
    7. Hyungjin Kim & Chang Min Park & Bhumsuk Keam & Sang Joon Park & Miso Kim & Tae Min Kim & Dong-Wan Kim & Dae Seog Heo & Jin Mo Goo, 2017. "The prognostic value of CT radiomic features for patients with pulmonary adenocarcinoma treated with EGFR tyrosine kinase inhibitors," PLOS ONE, Public Library of Science, vol. 12(11), pages 1-13, November.
    8. Joe Hirschberg and Jenny Lye, 2009. "Notes on the Construction of Geometric Representations of Confidence Intervals of Ratios using Stata, Gauss and Eviews," Department of Economics - Working Papers Series 1061, The University of Melbourne.
    9. Alicja Jokiel-Rokita & Rafał Topolnicki, 2019. "Minimum distance estimation of the binormal ROC curve," Statistical Papers, Springer, vol. 60(6), pages 2161-2183, December.
    10. Douglas G. Bonett & Robert M. Price Jr., 2020. "Confidence Intervals for Ratios of Means and Medians," Journal of Educational and Behavioral Statistics, , vol. 45(6), pages 750-770, December.
    11. Lukas Leitner, 2024. "Imprecision in the Estimation of Willingness to Pay Using Subjective Well-Being Data," Journal of Happiness Studies, Springer, vol. 25(7), pages 1-40, October.
    12. Sean F. Reardon & Benjamin R. Shear & Katherine E. Castellano & Andrew D. Ho, 2017. "Using Heteroskedastic Ordered Probit Models to Recover Moments of Continuous Test Score Distributions From Coarsened Data," Journal of Educational and Behavioral Statistics, , vol. 42(1), pages 3-45, February.
    13. Wang, Suohong & Zhang, Biao, 2014. "Semiparametric empirical likelihood confidence intervals for AUC under a density ratio model," Computational Statistics & Data Analysis, Elsevier, vol. 70(C), pages 101-115.
    14. Peng Wang & Siqi Xu & Yi‐Xin Wang & Baolin Wu & Wing Kam Fung & Guimin Gao & Zhijiang Liang & Nianjun Liu, 2021. "Penalized Fieller's confidence interval for the ratio of bivariate normal means," Biometrics, The International Biometric Society, vol. 77(4), pages 1355-1368, December.
    15. Mehlum, Halvor, 2017. "A polar confidence curve applied to Fieller’s ratios," Memorandum 05/2017, Oslo University, Department of Economics.
    16. Yang, Hanfang & Zhao, Yichuan, 2012. "Smoothed empirical likelihood for ROC curves with censored data," Journal of Multivariate Analysis, Elsevier, vol. 109(C), pages 254-263.
    17. Yang, Hanfang & Zhao, Yichuan, 2015. "Smoothed jackknife empirical likelihood inference for ROC curves with missing data," Journal of Multivariate Analysis, Elsevier, vol. 140(C), pages 123-138.
    18. Wan, Shuwen & Zhang, Biao, 2008. "Comparing correlated ROC curves for continuous diagnostic tests under density ratio models," Computational Statistics & Data Analysis, Elsevier, vol. 53(1), pages 233-245, September.
    19. Lang, Joseph B., 1999. "Bayesian ordinal and binary regression models with a parametric family of mixture links," Computational Statistics & Data Analysis, Elsevier, vol. 31(1), pages 59-87, July.
    20. Zhong Guan & Cheng Peng, 2011. "A rank-based empirical likelihood approach to two-sample proportional odds model and its goodness of fit," Journal of Nonparametric Statistics, Taylor & Francis Journals, vol. 23(3), pages 763-780.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0221433. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.