IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v78y2014icp69-81.html
   My bibliography  Save this article

Three-group ROC analysis: A nonparametric predictive approach

Author

Listed:
  • Coolen-Maturi, Tahani
  • Elkhafifi, Faiza F.
  • Coolen, Frank P.A.

Abstract

Measuring the accuracy of diagnostic tests is crucial in many application areas, in particular medicine and health care. The receiver operating characteristic (ROC) surface is a useful tool to assess the ability of a diagnostic test to discriminate among three ordered classes or groups. Nonparametric predictive inference (NPI) is a frequentist statistical method that is explicitly aimed at using few modelling assumptions in addition to data, enabled through the use of lower and upper probabilities to quantify uncertainty. It focuses exclusively on a future observation, which may be particularly relevant if one considers decisions about a diagnostic test to be applied to a future patient. The NPI approach to three-group ROC analysis is presented, including results on the volumes under the ROC surfaces and choice of decision threshold for the diagnosis.

Suggested Citation

  • Coolen-Maturi, Tahani & Elkhafifi, Faiza F. & Coolen, Frank P.A., 2014. "Three-group ROC analysis: A nonparametric predictive approach," Computational Statistics & Data Analysis, Elsevier, vol. 78(C), pages 69-81.
    • Handle: RePEc:eee:csdana:v:78:y:2014:i:c:p:69-81
    DOI: 10.1016/j.csda.2014.04.005
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167947314001017
    Download Restriction: Full text for ScienceDirect subscribers only.
    File URL: https://libkey.io/10.1016/j.csda.2014.04.005?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Chen, Weijie & Yousef, Waleed A. & Gallas, Brandon D. & Hsu, Elizabeth R. & Lababidi, Samir & Tang, Rong & Pennello, Gene A. & Symmans, W. Fraser & Pusztai, Lajos, 2012. "Uncertainty estimation with a finite dataset in the assessment of classification models," Computational Statistics & Data Analysis, Elsevier, vol. 56(5), pages 1016-1027.
    2. Zhang, Biao, 2006. "A semiparametric hypothesis testing procedure for the ROC curve area under a density ratio model," Computational Statistics & Data Analysis, Elsevier, vol. 50(7), pages 1855-1876, April.
    3. Waegeman, Willem & De Baets, Bernard & Boullart, Luc, 2008. "On the scalability of ordered multi-class ROC analysis," Computational Statistics & Data Analysis, Elsevier, vol. 52(7), pages 3371-3388, March.
    4. J. F. Lawless & Marc Fredette, 2005. "Frequentist prediction intervals and predictive distributions," Biometrika, Biometrika Trust, vol. 92(3), pages 529-542, September.
    5. Rodríguez-Álvarez, María Xosé & Roca-Pardiñas, Javier & Cadarso-Suárez, Carmen, 2011. "A new flexible direct ROC regression model: Application to the detection of cardiovascular risk factors by anthropometric measures," Computational Statistics & Data Analysis, Elsevier, vol. 55(12), pages 3257-3270, December.
    6. Adler, Werner & Lausen, Berthold, 2009. "Bootstrap estimated true and false positive rates and ROC curve," Computational Statistics & Data Analysis, Elsevier, vol. 53(3), pages 718-729, January.
    7. Rodríguez-Álvarez, María Xosé & Tahoces, Pablo G. & Cadarso-Suárez, Carmen & Lado, María José, 2011. "Comparative study of ROC regression techniques--Applications for the computer-aided diagnostic system in breast cancer detection," Computational Statistics & Data Analysis, Elsevier, vol. 55(1), pages 888-902, January.
    8. Lopez-de-Ullibarri, Ignacio & Cao, Ricardo & Cadarso-Suarez, Carmen & Lado, Maria J., 2008. "Nonparametric estimation of conditional ROC curves: Application to discrimination tasks in computerized detection of early breast cancer," Computational Statistics & Data Analysis, Elsevier, vol. 52(5), pages 2623-2631, January.
    9. Kang, Le & Tian, Lili, 2013. "Estimation of the volume under the ROC surface with three ordinal diagnostic categories," Computational Statistics & Data Analysis, Elsevier, vol. 62(C), pages 39-51.
    10. Airola, Antti & Pahikkala, Tapio & Waegeman, Willem & De Baets, Bernard & Salakoski, Tapio, 2011. "An experimental comparison of cross-validation techniques for estimating the area under the ROC curve," Computational Statistics & Data Analysis, Elsevier, vol. 55(4), pages 1828-1844, April.
    11. Stylianos Z. Xanthopoulos & Christos T. Nakas, 2007. "A generalized ROC approach for the validation of credit rating systems and scorecards," Journal of Risk Finance, Emerald Group Publishing, vol. 8(5), pages 481-488, November.
    12. Lai, Chin-Ying & Tian, Lili & Schisterman, Enrique F., 2012. "Exact confidence interval estimation for the Youden index and its corresponding optimal cut-point," Computational Statistics & Data Analysis, Elsevier, vol. 56(5), pages 1103-1114.
    13. Christos T. Nakas & Todd A. Alonzo, 2007. "ROC Graphs for Assessing the Ability of a Diagnostic Marker to Detect Three Disease Classes with an Umbrella Ordering," Biometrics, The International Biometric Society, vol. 63(2), pages 603-609, June.
    14. Yousef, Waleed A. & Kundu, Subrata & Wagner, Robert F., 2009. "Nonparametric estimation of the threshold at an operating point on the ROC curve," Computational Statistics & Data Analysis, Elsevier, vol. 53(12), pages 4370-4383, October.
    15. Wan, Shuwen & Zhang, Biao, 2008. "Comparing correlated ROC curves for continuous diagnostic tests under density ratio models," Computational Statistics & Data Analysis, Elsevier, vol. 53(1), pages 233-245, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Pardo-Fernandez, Juan Carlos & Rodriguez-alvarez, Maria Xose & Van Keilegom, Ingrid, 2013. "A review on ROC curves in the presence of covariates," LIDAM Discussion Papers ISBA 2013050, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    2. Rodríguez-Álvarez, María Xosé & Roca-Pardiñas, Javier & Cadarso-Suárez, Carmen, 2011. "A new flexible direct ROC regression model: Application to the detection of cardiovascular risk factors by anthropometric measures," Computational Statistics & Data Analysis, Elsevier, vol. 55(12), pages 3257-3270, December.
    3. Chen, Zhelun & O’Neill, Zheng & Wen, Jin & Pradhan, Ojas & Yang, Tao & Lu, Xing & Lin, Guanjing & Miyata, Shohei & Lee, Seungjae & Shen, Chou & Chiosa, Roberto & Piscitelli, Marco Savino & Capozzoli, , 2023. "A review of data-driven fault detection and diagnostics for building HVAC systems," Applied Energy, Elsevier, vol. 339(C).
    4. Acharki, Naoufal & Bertoncello, Antoine & Garnier, Josselin, 2023. "Robust prediction interval estimation for Gaussian processes by cross-validation method," Computational Statistics & Data Analysis, Elsevier, vol. 178(C).
    5. Jorge Navarro & Francesco Buono, 2023. "Predicting future failure times by using quantile regression," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 86(5), pages 543-576, July.
    6. Yuanyuan Shen & Katherine P. Liao & Tianxi Cai, 2015. "Sparse kernel machine regression for ordinal outcomes," Biometrics, The International Biometric Society, vol. 71(1), pages 63-70, March.
    7. Wang, Hsiuying, 2008. "Coverage probability of prediction intervals for discrete random variables," Computational Statistics & Data Analysis, Elsevier, vol. 53(1), pages 17-26, September.
    8. Vidoni, Paolo, 2015. "Calibrated multivariate distributions for improved conditional prediction," Journal of Multivariate Analysis, Elsevier, vol. 142(C), pages 16-25.
    9. Haojin Zhou & Tapan Nayak, 2015. "On the equivariance criterion in statistical prediction," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 67(3), pages 541-555, June.
    10. Paolo Vidoni, 2017. "Improved multivariate prediction regions for Markov process models," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 26(1), pages 1-18, March.
    11. Cheam, Amay S.M. & McNicholas, Paul D., 2016. "Modelling receiver operating characteristic curves using Gaussian mixtures," Computational Statistics & Data Analysis, Elsevier, vol. 93(C), pages 192-208.
    12. Foucher Yohann & Danger Richard, 2012. "Time Dependent ROC Curves for the Estimation of True Prognostic Capacity of Microarray Data," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 11(6), pages 1-22, November.
    13. V. J. Roelofs & F. P. A. Coolen & A. D. M. Hart, 2011. "Nonparametric Predictive Inference for Exposure Assessment," Risk Analysis, John Wiley & Sons, vol. 31(2), pages 218-227, February.
    14. Mogensen, Ulla B. & Ishwaran, Hemant & Gerds, Thomas A., 2012. "Evaluating Random Forests for Survival Analysis Using Prediction Error Curves," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 50(i11).
    15. De Oliveira, Victor & Kone, Bazoumana, 2015. "Prediction intervals for integrals of Gaussian random fields," Computational Statistics & Data Analysis, Elsevier, vol. 83(C), pages 37-51.
    16. Paolo Vidoni, 2009. "A simple procedure for computing improved prediction intervals for autoregressive models," Journal of Time Series Analysis, Wiley Blackwell, vol. 30(6), pages 577-590, November.
    17. Omar M. Bdair & Mohammad Z. Raqab, 2022. "Prediction of future censored lifetimes from mixture exponential distribution," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 85(7), pages 833-857, October.
    18. Poon, Wai-Yin & Qiu, Shi-Fang & Tang, Man-Lai, 2015. "Confidence interval construction for the Youden index based on partially validated series," Computational Statistics & Data Analysis, Elsevier, vol. 84(C), pages 116-134.
    19. Doyo Gragn Enki & Angela Noufaily & Paddy Farrington & Paul Garthwaite & Nick Andrews & Andre Charlett, 2017. "Taylor's power law and the statistical modelling of infectious disease surveillance data," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 180(1), pages 45-72, January.
    20. Seyed Poorya Mirfallah Lialestani & David Parcerisa & Mahjoub Himi & Abbas Abbaszadeh Shahri, 2022. "Generating 3D Geothermal Maps in Catalonia, Spain Using a Hybrid Adaptive Multitask Deep Learning Procedure," Energies, MDPI, vol. 15(13), pages 1-16, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:78:y:2014:i:c:p:69-81. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.