IDEAS home Printed from https://ideas.repec.org/p/imf/imfwpa/2012-122.html
   My bibliography  Save this paper

Appraising Credit Ratings: Does the CAP Fit Better than the ROC?

Author

Listed:
  • Mr. R. John Irwin
  • Mr. Timothy C Irwin

Abstract

ROC and CAP analysis are alternative methods for evaluating a wide range of diagnostic systems, including assessments of credit risk. ROC analysis is widely used in many fields, but in finance CAP analysis is more common. We compare the two methods, using as an illustration the ability of the OECD’s country risk ratings to predict whether a country will have a program with the IMF (an indicator of financial distress). ROC and CAP analyses both have the advantage of generating measures of accuracy that are independent of the choice of diagnostic threshold, such as risk rating. ROC analysis has other beneficial features, including theories for fitting models to data and for setting the optimal threshold, that we show could also be incorporated into CAP analysis. But the natural interpretation of the ROC measure of accuracy and the independence of ROC curves from the probability of default are advantages unavailable to CAP analysis.

Suggested Citation

  • Mr. R. John Irwin & Mr. Timothy C Irwin, 2012. "Appraising Credit Ratings: Does the CAP Fit Better than the ROC?," IMF Working Papers 2012/122, International Monetary Fund.
    • Handle: RePEc:imf:imfwpa:2012/122
    as

    Download full text from publisher

    File URL: http://www.imf.org/external/pubs/cat/longres.aspx?sk=25910
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Edward Altman & Gabriele Sabato, 2005. "Effects of the New Basel Capital Accord on Bank Capital Requirements for SMEs," Journal of Financial Services Research, Springer;Western Finance Association, vol. 28(1), pages 15-42, October.
    2. Miss Gabriela Dobrescu & Iva Petrova & Nazim Belhocine & Mr. Emanuele Baldacci, 2011. "Assessing Fiscal Stress," IMF Working Papers 2011/100, International Monetary Fund.
    3. Stein, Roger M., 2005. "The relationship between default prediction and lending profits: Integrating ROC analysis and loan pricing," Journal of Banking & Finance, Elsevier, vol. 29(5), pages 1213-1236, May.
    4. Das, Sanjiv R. & Hanouna, Paul & Sarin, Atulya, 2009. "Accounting-based versus market-based cross-sectional models of CDS spreads," Journal of Banking & Finance, Elsevier, vol. 33(4), pages 719-730, April.
    5. Dirk Tasche, 2006. "Validation of internal rating systems and PD estimates," Papers physics/0606071, arXiv.org.
    6. Crouhy, Michel & Galai, Dan & Mark, Robert, 2000. "A comparative analysis of current credit risk models," Journal of Banking & Finance, Elsevier, vol. 24(1-2), pages 59-117, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Turalay Kenc & Emrah Ismail Cevik & Sel Dibooglu, 2021. "Bank default indicators with volatility clustering," Annals of Finance, Springer, vol. 17(1), pages 127-151, March.
    2. Modina, Michele & Pietrovito, Filomena & Gallucci, Carmen & Formisano, Vincenzo, 2023. "Predicting SMEs’ default risk: Evidence from bank-firm relationship data," The Quarterly Review of Economics and Finance, Elsevier, vol. 89(C), pages 254-268.
    3. Renatas Špicas & Airidas Neifaltas & Rasa Kanapickienė & Greta Keliuotytė-Staniulėnienė & Deimantė Vasiliauskaitė, 2023. "Estimating the Acceptance Probabilities of Consumer Loan Offers in an Online Loan Comparison and Brokerage Platform," Risks, MDPI, vol. 11(7), pages 1-30, July.
    4. McCarthy, Patrick, 2024. "Predicting trips to health care facilities: A binary logit and receiver operating characteristics (ROC) approach," Research in Transportation Economics, Elsevier, vol. 103(C).
    5. Péter Bauer & Marianna Endrész, 2016. "Modelling Bankruptcy Using Hungarian Firm-Level Data," MNB Occasional Papers 2016/122, Magyar Nemzeti Bank (Central Bank of Hungary).
    6. Nehrebecka Natalia, 2018. "An Evaluation of the Discriminatory Power of Selected Polish Bankruptcy Prediction Models As Part of the Validation Process," Financial Sciences. Nauki o Finansach, Sciendo, vol. 23(4), pages 63-88, December.
    7. Suleiman A. Badayi & Bolaji T. Matemilola & Bany‐Ariffin A.N & Lau Wei Theng, 2021. "Does corporate social responsibility influence firm probability of default?," International Journal of Finance & Economics, John Wiley & Sons, Ltd., vol. 26(3), pages 3377-3395, July.
    8. Pranith K. Roy & Krishnendu Shaw, 2023. "A credit scoring model for SMEs using AHP and TOPSIS," International Journal of Finance & Economics, John Wiley & Sons, Ltd., vol. 28(1), pages 372-391, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Francesco Dainelli & Francesco Giunta & Fabrizio Cipollini, 2013. "Determinants of SME credit worthiness under Basel rules: the value of credit history information," PSL Quarterly Review, Economia civile, vol. 66(264), pages 21-47.
    2. Giulio Bottazzi & Marco Grazzi & Angelo Secchi & Federico Tamagni, 2011. "Financial and economic determinants of firm default," Journal of Evolutionary Economics, Springer, vol. 21(3), pages 373-406, August.
    3. Maria Correia & Johnny Kang & Scott Richardson, 2018. "Asset volatility," Review of Accounting Studies, Springer, vol. 23(1), pages 37-94, March.
    4. Raffaella Calabrese, 2011. "Cost-sensitive classification for rare events: an application to the credit rating model validation for SMEs," Working Papers 201134, Geary Institute, University College Dublin.
    5. Abinzano, Isabel & Gonzalez-Urteaga, Ana & Muga, Luis & Sanchez, Santiago, 2020. "Performance of default-risk measures: the sample matters," Journal of Banking & Finance, Elsevier, vol. 120(C).
    6. Correia, Maria & Kang, Johnny & Richardson, Scott, 2018. "Asset volatility," LSE Research Online Documents on Economics 84405, London School of Economics and Political Science, LSE Library.
    7. Raffaella Calabrese, 2012. "Improving Classifier Performance Assessment of Credit Scoring Models," Working Papers 201204, Geary Institute, University College Dublin.
    8. Suzan Hol, 2006. "The influence of the business cycle on bankruptcy probability," Discussion Papers 466, Statistics Norway, Research Department.
    9. Pesaran, M. Hashem & Schuermann, Til & Treutler, Bjorn-Jakob & Weiner, Scott M., 2006. "Macroeconomic Dynamics and Credit Risk: A Global Perspective," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 38(5), pages 1211-1261, August.
    10. Berry K. Wilson & John T. Donnellan, 2016. "The Technology of Ratings Then and Now; Hiding in Plain Sight," Financial Markets, Institutions & Instruments, John Wiley & Sons, vol. 25(1), pages 49-74, January.
    11. Hsin-Hui Chiu & Eva Wagner, 2020. "CEO Bonus Pay and Firm Credit Risk," International Journal of Risk and Contingency Management (IJRCM), IGI Global, vol. 9(1), pages 1-19, January.
    12. Tang, Dragon Yongjun & Yan, Hong, 2017. "Understanding transactions prices in the credit default swaps market," Journal of Financial Markets, Elsevier, vol. 32(C), pages 1-27.
    13. Chiara Pederzoli & Grid Thoma & Costanza Torricelli, 2013. "Modelling Credit Risk for Innovative SMEs: the Role of Innovation Measures," Journal of Financial Services Research, Springer;Western Finance Association, vol. 44(1), pages 111-129, August.
    14. Kanak Patel & Ricardo Pereira, 2007. "Expected Default Probabilities in Structural Models: Empirical Evidence," The Journal of Real Estate Finance and Economics, Springer, vol. 34(1), pages 107-133, January.
    15. Mikkel Hermansen & Oliver Röhn, 2017. "Economic resilience: The usefulness of early warning indicators in OECD countries," OECD Journal: Economic Studies, OECD Publishing, vol. 2016(1), pages 9-35.
    16. Georges Dionne, 2003. "The Foundationsof Banks' Risk Regulation: A Review of Literature," THEMA Working Papers 2003-46, THEMA (THéorie Economique, Modélisation et Applications), Université de Cergy-Pontoise.
    17. Ephraim Clark & Geeta Lakshmi, 2003. "Controlling the risk: a case study of the Indian liquidity crisis 1990-92," Journal of International Development, John Wiley & Sons, Ltd., vol. 15(3), pages 285-298.
    18. Jorge M. Uribe, 2023. ""Fiscal crises and climate change"," IREA Working Papers 202303, University of Barcelona, Research Institute of Applied Economics, revised Feb 2023.
    19. Meng-Jou Lu & Cathy Yi-Hsuan Chen & Wolfgang Karl Härdle, 2017. "Copula-based factor model for credit risk analysis," Review of Quantitative Finance and Accounting, Springer, vol. 49(4), pages 949-971, November.
    20. Huang, Hsing-Hua & Lee, Han-Hsing, 2013. "Product market competition and credit risk," Journal of Banking & Finance, Elsevier, vol. 37(2), pages 324-340.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:imf:imfwpa:2012/122. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Akshay Modi (email available below). General contact details of provider: https://edirc.repec.org/data/imfffus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.