IDEAS home Printed from https://ideas.repec.org/a/tsj/stataj/v9y2009i1p1-16.html
   My bibliography  Save this article

Estimation and comparison of receiver operating characteristic curves

Author

Listed:
  • Margaret S. Pepe

    (Fred Hutchinson Cancer Research Center)

  • Gary Longton

    (Fred Hutchinson Cancer Research Center)

  • Holly Janes

    (Fred Hutchinson Cancer Research Center)

Abstract

The receiver operating characteristic (ROC) curve displays the capacity of a marker or diagnostic test to discriminate between two groups of subjects, cases versus controls. We present a comprehensive suite of Stata commands for performing ROC analysis. Nonparametric, semiparametric, and parametric estimators are calculated. Comparisons between curves are based on the area or partial area under the ROC curve. Alternatively, pointwise comparisons between ROC curves or inverse ROC curves can be made. We describe options to adjust these analyses for covariates and to perform ROC regression in a companion article. We use a unified framework by representing the ROC curve as the distribution of the marker in cases where we have standardized it to the control reference distribution. Copyright 2009 by StataCorp LP.

Suggested Citation

  • Margaret S. Pepe & Gary Longton & Holly Janes, 2009. "Estimation and comparison of receiver operating characteristic curves," Stata Journal, StataCorp LP, vol. 9(1), pages 1-16, March.
    • Handle: RePEc:tsj:stataj:v:9:y:2009:i:1:p:1-16
    as

    Download full text from publisher

    File URL: http://www.stata-journal.com/article.html?article=st0154
    Download Restriction: no
    File URL: http://www.stata-journal.com/software/sj9-1/st0154/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Lori E. Dodd & Margaret S. Pepe, 2003. "Partial AUC Estimation and Regression," Biometrics, The International Biometric Society, vol. 59(3), pages 614-623, September.
    2. Margaret Sullivan Pepe & Tianxi Cai, 2004. "The Analysis of Placement Values for Evaluating Discriminatory Measures," Biometrics, The International Biometric Society, vol. 60(2), pages 528-535, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Drehmann, Mathias & Juselius, Mikael, 2014. "Evaluating early warning indicators of banking crises: Satisfying policy requirements," International Journal of Forecasting, Elsevier, vol. 30(3), pages 759-780.
    2. John Muschelli, 2020. "ROC and AUC with a Binary Predictor: a Potentially Misleading Metric," Journal of Classification, Springer;The Classification Society, vol. 37(3), pages 696-708, October.
    3. Jesús F. Lampón & Pablo Cabanelas-Lorenzo & Santiago Lago-Peñas, 2013. "Why firms relocate their production overseas? The answer lies inside: corporate, logistic and technological determinants," Working Papers 2013/3, Institut d'Economia de Barcelona (IEB).
    4. Jung Wha Chung & Beom Hee Kim & Chung Seop Lee & Gi Hyun Kim & Hyung Rae Sohn & Bo Young Min & Joon Chang Song & Hyun Kyung Park & Eun Sun Jang & Hyuk Yoon & Jaihwan Kim & Cheol Min Shin & Young Soo P, 2016. "Optimizing Surveillance Performance of Alpha-Fetoprotein by Selection of Proper Target Population in Chronic Hepatitis B," PLOS ONE, Public Library of Science, vol. 11(12), pages 1-13, December.
    5. André K. Anundsen & Karsten Gerdrup & Frank Hansen & Kasper Kragh‐Sørensen, 2016. "Bubbles and Crises: The Role of House Prices and Credit," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 31(7), pages 1291-1311, November.
    6. Detken, Carsten & Weeken, Olaf & Alessi, Lucia & Bonfim, Diana & Boucinha, Miguel & Castro, Christian & Frontczak, Sebastian & Giordana, Gaston & Giese, Julia & Wildmann, Nadya & Kakes, Jan & Klaus, B, 2014. "Operationalising the countercyclical capital buffer: indicator selection, threshold identification and calibration options," ESRB Occasional Paper Series 5, European Systemic Risk Board.
    7. Aastveit, Knut Are & Anundsen, André K. & Herstad, Eyo I., 2019. "Residential investment and recession predictability," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1790-1799.
    8. Elizabeth Gutierrez & Jake Krupa & Miguel Minutti-Meza & Maria Vulcheva, 2020. "Do going concern opinions provide incremental information to predict corporate defaults?," Review of Accounting Studies, Springer, vol. 25(4), pages 1344-1381, December.
    9. Holly Janes & Gary Longton & Margaret S. Pepe, 2009. "Accommodating covariates in receiver operating characteristic analysis," Stata Journal, StataCorp LP, vol. 9(1), pages 17-39, March.
    10. Xinyue Wu & Hong Zhu & Liuru Hu & Jian Meng & Fulu Sun, 2024. "Analysis of Short-Term Heavy Rainfall-Based Urban Flood Disaster Risk Assessment Using Integrated Learning Approach," Sustainability, MDPI, vol. 16(18), pages 1-19, September.
    11. Oke Gerke & Antonia Zapf, 2022. "Convergence Behavior of Optimal Cut-Off Points Derived from Receiver Operating Characteristics Curve Analysis: A Simulation Study," Mathematics, MDPI, vol. 10(22), pages 1-14, November.
    12. Bespalova, Olga, 2018. "Forecast Evaluation in Macroeconomics and International Finance. Ph.D. thesis, George Washington University, Washington, DC, USA," MPRA Paper 117706, University Library of Munich, Germany.
    13. Jesús F. Lampón & Pablo Cabanelas-Lorenzo & Santiago Lago-Peñas, 2013. "Why firms relocate their production overseas? The answer lies inside: corporate, logistic and technological determinants," Working Papers 2013/3, Institut d'Economia de Barcelona (IEB).
    14. Mund, Carolin & Neuhäusler, Peter, 2015. "Towards an early-stage identification of emerging topics in science—The usability of bibliometric characteristics," Journal of Informetrics, Elsevier, vol. 9(4), pages 1018-1033.
    15. Marcin Łupiński, 2019. "Wskaźniki wczesnego ostrzegania przed niestabilnością finansową polskiego sektora bankowego," Collegium of Economic Analysis Annals, Warsaw School of Economics, Collegium of Economic Analysis, issue 55, pages 99-113.
    16. Geršl, Adam & Jašová, Martina, 2018. "Credit-based early warning indicators of banking crises in emerging markets," Economic Systems, Elsevier, vol. 42(1), pages 18-31.
    17. Nancy Puttkammer & Steven Zeliadt & Jean Gabriel Balan & Janet Baseman & Rodney Destiné & Jean Wysler Domerçant & Garilus France & Nathaelf Hyppolite & Valérie Pelletier & Nernst Atwood Raphael & Kenn, 2014. "Development of an Electronic Medical Record Based Alert for Risk of HIV Treatment Failure in a Low-Resource Setting," PLOS ONE, Public Library of Science, vol. 9(11), pages 1-12, November.
    18. Rocío Aznar-Gimeno & Luis M. Esteban & Gorka Labata-Lezaun & Rafael del-Hoyo-Alonso & David Abadia-Gallego & J. Ramón Paño-Pardo & M. José Esquillor-Rodrigo & Ángel Lanas & M. Trinidad Serrano, 2021. "A Clinical Decision Web to Predict ICU Admission or Death for Patients Hospitalised with COVID-19 Using Machine Learning Algorithms," IJERPH, MDPI, vol. 18(16), pages 1-20, August.
    19. Eva V Applebaum & Dominic Breton & Zhuo Wei Feng & An-Tchi Ta & Kayley Walsh & Kathleen Chassé & Shawn M Robbins, 2017. "Modified 30-second Sit to Stand test predicts falls in a cohort of institutionalized older veterans," PLOS ONE, Public Library of Science, vol. 12(5), pages 1-13, May.
    20. Carsten Detken & Olaf Weeken & Lucia Alessi & Diana Bonfim & Miguel M. Boucinha & Christian Castro & Sebastian Frontczak & Gaston Giordana & Julia Giese & Nadya Jahn & Jan Kakes & Benjamin Klaus & Jan, 2014. "Operationalising the countercyclical capital buffer: indicator selection, threshold identification and calibration options," ESRB Occasional Paper Series 05, European Systemic Risk Board.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Holly Janes & Gary Longton & Margaret S. Pepe, 2009. "Accommodating covariates in receiver operating characteristic analysis," Stata Journal, StataCorp LP, vol. 9(1), pages 17-39, March.
    2. Tianxi Cai & Yingye Zheng, 2007. "Model Checking for ROC Regression Analysis," Biometrics, The International Biometric Society, vol. 63(1), pages 152-163, March.
    3. Soutik Ghosal & Zhen Chen, 2022. "Discriminatory Capacity of Prenatal Ultrasound Measures for Large-for-Gestational-Age Birth: A Bayesian Approach to ROC Analysis Using Placement Values," Statistics in Biosciences, Springer;International Chinese Statistical Association, vol. 14(1), pages 1-22, April.
    4. Margaret Sullivan Pepe & Tianxi Cai, 2004. "The Analysis of Placement Values for Evaluating Discriminatory Measures," Biometrics, The International Biometric Society, vol. 60(2), pages 528-535, June.
    5. Elisa–María Molanes-López & Ricardo Cao, 2008. "Relative density estimation for left truncated and right censored data," Journal of Nonparametric Statistics, Taylor & Francis Journals, vol. 20(8), pages 693-720.
    6. Chen, Xiwei & Vexler, Albert & Markatou, Marianthi, 2015. "Empirical likelihood ratio confidence interval estimation of best linear combinations of biomarkers," Computational Statistics & Data Analysis, Elsevier, vol. 82(C), pages 186-198.
    7. Mei-Cheng Wang & Shanshan Li, 2012. "Bivariate Marker Measurements and ROC Analysis," Biometrics, The International Biometric Society, vol. 68(4), pages 1207-1218, December.
    8. Merve Basol & Dincer Goksuluk & Ergun Karaagaoglu, 2023. "Comparing the diagnostic performance of methods used in a full-factorial design multi-reader multi-case studies," Computational Statistics, Springer, vol. 38(3), pages 1537-1553, September.
    9. Jialiang Li & Jason P. Fine, 2010. "Weighted area under the receiver operating characteristic curve and its application to gene selection," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 59(4), pages 673-692, August.
    10. Wang, Suohong & Zhang, Biao, 2014. "Semiparametric empirical likelihood confidence intervals for AUC under a density ratio model," Computational Statistics & Data Analysis, Elsevier, vol. 70(C), pages 101-115.
    11. Ángel Beade & Manuel Rodríguez & José Santos, 2024. "Business failure prediction models with high and stable predictive power over time using genetic programming," Operational Research, Springer, vol. 24(3), pages 1-41, September.
    12. Gunther Glehr & Paloma Riquelme & Katharina Kronenberg & Robert Lohmayer & Víctor J. López-Madrona & Michael Kapinsky & Hans J. Schlitt & Edward K. Geissler & Rainer Spang & Sebastian Haferkamp & Jame, 2024. "Restricting datasets to classifiable samples augments discovery of immune disease biomarkers," Nature Communications, Nature, vol. 15(1), pages 1-21, December.
    13. Gengsheng Qin & Xiao-Hua Zhou, 2006. "Empirical Likelihood Inference for the Area under the ROC Curve," Biometrics, The International Biometric Society, vol. 62(2), pages 613-622, June.
    14. Letón, Emilio & Molanes, Elisa M., 2009. "Adjusted empirical likelihood estimation of the youden index and associated threshold for the bigamma model," DES - Working Papers. Statistics and Econometrics. WS ws091907, Universidad Carlos III de Madrid. Departamento de Estadística.
    15. Sergio Picart-Armada & Steven J Barrett & David R Willé & Alexandre Perera-Lluna & Alex Gutteridge & Benoit H Dessailly, 2019. "Benchmarking network propagation methods for disease gene identification," PLOS Computational Biology, Public Library of Science, vol. 15(9), pages 1-24, September.
    16. Xiao Song & Xiao-Hua Zhou, 2004. "A Marginal Model Approach for Analysis of Multi-reader Multi-test Receiver Operating Characteristic (ROC) Data," UW Biostatistics Working Paper Series 1067, Berkeley Electronic Press.
    17. Y. Huang & M. S. Pepe, 2009. "A Parametric ROC Model-Based Approach for Evaluating the Predictiveness of Continuous Markers in Case–Control Studies," Biometrics, The International Biometric Society, vol. 65(4), pages 1133-1144, December.
    18. Peterson, A. Townsend & Papeş, Monica & Soberón, Jorge, 2008. "Rethinking receiver operating characteristic analysis applications in ecological niche modeling," Ecological Modelling, Elsevier, vol. 213(1), pages 63-72.
    19. Man-Jen Hsu & Huey-Miin Hsueh, 2013. "The linear combinations of biomarkers which maximize the partial area under the ROC curves," Computational Statistics, Springer, vol. 28(2), pages 647-666, April.
    20. Rodríguez-Álvarez, María Xosé & Roca-Pardiñas, Javier & Cadarso-Suárez, Carmen, 2011. "A new flexible direct ROC regression model: Application to the detection of cardiovascular risk factors by anthropometric measures," Computational Statistics & Data Analysis, Elsevier, vol. 55(12), pages 3257-3270, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:tsj:stataj:v:9:y:2009:i:1:p:1-16. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Christopher F. Baum or Lisa Gilmore (email available below). General contact details of provider: http://www.stata-journal.com/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.