The distribution of defaults and Bayesian model validation

ABSTRACT Quantitative rating systems are increasingly being used for the purposes of capital allocation and pricing credits. For these purposes, it is important to validate the accuracy of the probability of default (PD) estimates generated by the rating system and not merely focus on evaluating the discriminatory power of the system. The validation of the accuracy of the PD quantification has been a challenge, fraught with theoretical difficulties (mainly, the impact of correlation) and data issues (eg, the infrequency of default events). Moreover, models – even “correct” models – will over-predict default rates most of the time. Working within the standard single-factor framework, we present two Bayesian approaches to the level validation of a PD model. The first approach provides a set of techniques to facilitate risk assessment in the absence of sufficient historical default data. It derives the posterior distribution of a PD, given zero realized defaults, thereby providing a framework for determining the upper bound for a PD in relation to a low default portfolio. The second approach provides a means for monitoring the calibration of a rating system. It derives the posterior distribution of the aggregate shock in the macro-economic environment, given a realized default rate. By comparing this distribution to the institution’s view of the stage of the credit cycle its borrowers are in, this approach provides useful insight for whether an institution should revisit the calibration of its rating system. This method allows one to determine that a calibration needs to be revisited even when the default rate is within the 95% confidence level computed under the standard approach.