IDEAS home Printed from https://ideas.repec.org/p/ecb/ecbwps/20161885.html
   My bibliography  Save this paper

Advances in multivariate back-testing for credit risk underestimation

Author

Listed:
  • Sauer, Stephan
  • Coppens, François
  • Mayer, Manuel
  • Millischer, Laurent
  • Resch, Florian
  • Schulze, Klaas

Abstract

When back-testing the calibration quality of rating systems two-sided statistical tests can detect over- and underestimation of credit risk. Some users though, such as risk-averse investors and regulators, are primarily interested in the underestimation of risk only, and thus require one-sided tests. The established one-sided tests are multiple tests, which assess each rating class of the rating system separately and then combine the results to an overall assessment. However, these multiple tests may fail to detect underperformance of the whole rating system. Aiming to improve the overall assessment of rating systems, this paper presents a set of one-sided tests, which assess the performance of all rating classes jointly. These joint tests build on the method of Sterne [1954] for ranking possible outcomes by probability, which allows to extend back-testing to a setting of multiple rating classes. The new joint tests are compared to the most established one-sided multiple test and are further shown to outperform this benchmark in terms of power and size of the acceptance region. JEL Classification: C12, C52, G21, G24

Suggested Citation

  • Sauer, Stephan & Coppens, François & Mayer, Manuel & Millischer, Laurent & Resch, Florian & Schulze, Klaas, 2016. "Advances in multivariate back-testing for credit risk underestimation," Working Paper Series 1885, European Central Bank.
    • Handle: RePEc:ecb:ecbwps:20161885
    Note: 362683
    as

    Download full text from publisher

    File URL: https://www.ecb.europa.eu//pub/pdf/scpwps/ecbwp1885.en.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. François Coppens & Fernando Gonzáles & Gerhard Winkler, 2007. "The performance of credit rating systems in the assessment of collateral used in Eurosystem monetary policy operations," Working Paper Research 118, National Bank of Belgium.
    2. González, Fernando & Coppens, François & Winkler, Gerhard, 2007. "The performance of credit rating systems in the assessment of collateral used in Eurosystem monetary policy operations," Occasional Paper Series 65, European Central Bank.
    3. Jankowitsch, Rainer & Pichler, Stefan & Schwaiger, Walter S.A., 2007. "Modelling the economic value of credit rating systems," Journal of Banking & Finance, Elsevier, vol. 31(1), pages 181-198, January.
    4. Reiczigel, Jeno & Abonyi-Tóth, Zsolt & Singer, Júlia, 2008. "An exact confidence set for two binomial proportions and exact unconditional confidence intervals for the difference and ratio of proportions," Computational Statistics & Data Analysis, Elsevier, vol. 52(11), pages 5046-5053, July.
    5. Aussenegg, Wolfgang & Resch, Florian & Winkler, Gerhard, 2011. "Pitfalls and remedies in testing the calibration quality of rating systems," Journal of Banking & Finance, Elsevier, vol. 35(3), pages 698-708, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Patrick Kurth & Max Nendel & Jan Streicher, 2023. "A hypothesis test for the long-term calibration in rating systems with overlapping time windows," Papers 2312.14765, arXiv.org.
    2. António R. Antunes & Homero Gonçalves, 2016. "Firm default probabilities revisited," Economic Bulletin and Financial Stability Report Articles and Banco de Portugal Economic Studies, Banco de Portugal, Economics and Research Department.
    3. Dainelli, Francesco & Bet, Gianmarco & Fabrizi, Eugenio, 2024. "The financial health of a company and the risk of its default: Back to the future," International Review of Financial Analysis, Elsevier, vol. 95(PB).
    4. António Antunes & Homero Gonçalves & Pedro Prego, 2017. "Firm default probabilities revisited," IFC Bulletins chapters, in: Bank for International Settlements (ed.), Uses of central balance sheet data offices' information, volume 45, Bank for International Settlements.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mayerlen, Frank & Sola, Pierre & Be Duc, Louis, 2008. "The monetary presentation of the euro area balance of payments," Occasional Paper Series 96, European Central Bank.
    2. K. Barhoumi & S. Benk & R. Cristadoro & A. Den Reijer & A. Jakaitiene & P. Jelonek & A. Rua & K. Ruth & C. Van Nieuwenhuyze & G. Rünstler, 2008. "Short-term forecasting of GDP using large monthly datasets – A pseudo real-time forecast evaluation exercise," Working Paper Research 133, National Bank of Belgium.
    3. Blattner, Tobias Sebastian & Catenaro, Marco & Ehrmann, Michael & Strauch, Rolf & Turunen, Jarkko, 2008. "The predictability of monetary policy," Occasional Paper Series 83, European Central Bank.
    4. Thimann, Christian & Skala, Martin & Wölfinger, Regine, 2007. "The search for Columbus' egg: finding a new formula to determine quotas at the IMF," Occasional Paper Series 70, European Central Bank.
    5. Sturm, Michael & Adolf, Petra & Peschel, Dominik & Stráský, Jan, 2008. "The Gulf Cooperation Council countries: economic structures, recent developments and role in the global economy," Occasional Paper Series 92, European Central Bank.
    6. Roland Beck & Michael Fidora, 2008. "The impact of sovereign wealth funds on global financial markets," Intereconomics: Review of European Economic Policy, Springer;ZBW - Leibniz Information Centre for Economics;Centre for European Policy Studies (CEPS), vol. 43(6), pages 349-358, November.
    7. Winkler, Adalbert & Polański, Zbigniew, 2008. "Russia, EU enlargement and the euro," Occasional Paper Series 93, European Central Bank.
    8. Irina Bunda, 2007. "The Changing Role of the Exchange Rate in a Globalised Economy," Post-Print halshs-00372820, HAL.
    9. Russo, Daniela & Rosati, Simonetta & Chan, Diana & Fontan, Florence, 2007. "The securities custody industry," Occasional Paper Series 68, European Central Bank.
    10. Thierry Bracke & Matthieu Bussière & Michael Fidora & Roland Straub, 2010. "A Framework for Assessing Global Imbalances," The World Economy, Wiley Blackwell, vol. 33(9), pages 1140-1174, September.
    11. Sturm, Michael & Gurtner, François, 2007. "Fiscal policy in Mediterranean countries: developments, structures and implications for monetary policy," Occasional Paper Series 69, European Central Bank.
    12. De Santis, Roberto A. & Cappiello, Lorenzo & Baltzer, Markus & Manganelli, Simone, 2008. "Measuring financial integration in new EU Member States," Occasional Paper Series 81, European Central Bank.
    13. Stiegert, Roger & Leiner-Killinger, Nadine & López Pérez, Víctor & Vitale, Giovanni, 2007. "Structural reforms in EMU and the role of monetary policy: a survey of the literature," Occasional Paper Series 66, European Central Bank.
    14. Filippo Di Mauro & Katrin Forster, "undated". "Globalisation and the competitiveness of the Euro area," Working Papers 5, Department of the Treasury, Ministry of the Economy and of Finance.
    15. Gómez-Salvador, Ramón & Leiner-Killinger, Nadine, 2008. "An analysis of youth unemployment in the euro area," Occasional Paper Series 89, European Central Bank.
    16. Russo, Daniela & Caviglia, Giacomo & Papathanassiou, Chryssa & Rosati, Simonetta, 2007. "Prudential and oversight requirements for securities settlement," Occasional Paper Series 76, European Central Bank.
    17. Mehl, Arnaud & Bussière, Matthieu, 2008. "China's and India's roles in global trade and finance: twin titans for the new millennium?," Occasional Paper Series 80, European Central Bank.
    18. Moutot, Philippe & Jung, Alexander & Mongelli, Francesco Paolo, 2008. "The working of the eurosystem: monetary policy preparations and decision-making - selected issues," Occasional Paper Series 79, European Central Bank.
    19. Sánchez Muñoz, Carlos & Israël, Jean-Marc, 2007. "Towards harmonised balance of payments and international investment position statistics - the experience of the European compilers," Occasional Paper Series 67, European Central Bank.
    20. di Mauro, Filippo & Kaufmann, Robert K. & Karadeloglou, Pavlos, 2008. "Will oil prices decline over the long run?," Occasional Paper Series 98, European Central Bank.

    More about this item

    Keywords

    back-testing; credit ratings; one-sided; probability of default;
    All these keywords.

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • G21 - Financial Economics - - Financial Institutions and Services - - - Banks; Other Depository Institutions; Micro Finance Institutions; Mortgages
    • G24 - Financial Economics - - Financial Institutions and Services - - - Investment Banking; Venture Capital; Brokerage

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ecb:ecbwps:20161885. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Official Publications (email available below). General contact details of provider: https://edirc.repec.org/data/emieude.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.