Credit Risk Models - Do They Deliver Their Promises? A Quantitative Assessment

Abstract

We develop a framework to assess the statistical significance of expected default frequency as calculated by credit risk models. This framework is then used to analyze the quality of two commercially available models that have become popular among practitioners: KMV Credit Monitor and RiskCalc from Moody's. Using a unique database of expected default probability from both vendors, we study both the consistency of predictions and their timeliness. We introduce the concept of cumulative accuracy profile (CAP), which allows to see in one curve the percentage of companies whose defualts were captured by the models one year in advance. We also use the Miller's information test to see if the models add information to the S&P rating. The result of the analysis indicates that these models indeed add relevant information not accounted for by rating alone. Moreover, with respect to rating agencies, the models predict defaults more than ten months in advance on average.credit risk models, cumulative accuracy profile, risk modeling