Is it Worth the While? The Relevance of Qualitative Information in Credit Rating

25 Pages Posted: 24 Jun 2003

See all articles by Bina Lehmann

Bina Lehmann

Universitaet Konstanz - Center of Finance & Econometrics (CoFE)

Date Written: April 17, 2003

Abstract

This empirical study deals with the question whether soft facts (qualitative information, i.e. subjective judgments of credit analysts) considerably improve the forecast quality of bank internal credit ratings that are solely based on hard facts (financial ratios, checking account data).

An extensive sample (20.000 observations) of German SME credit data has been made available by a commercial bank to compare two models: one including, the other excluding qualitative information. Logistic regression is used to forecast default probabilities. The econometric quality as well as the classification and separation performance of the two models are assessed statistically and graphically, using various measures. It is shown that most of the widely used measures cannot be used for inter-sample comparison and offer therefore little informational content. Further, performance measures that are based on classification tables (i.e. a single cut-off value) should only be used if the (costs) benefits of (mis)classification are known. ROC (Receiver Operating Characteristic curve)-based measures and the inspection of ROC have been found to be the most useful criteria for model comparison. ROC inspection allows to compare models in a more qualitative way, adding information to the common inspection of numerical criteria.

The problem of observing very few defaults is solved by stratifying the estimation sample and using a re-sampling procedure. This results in empirical distributions of the performance measures. While, in similar studies, one (mean) value per measure and model is given, the existence of empirical distributions allows the difference between the two models to be quantified and to decide whether the models differ significantly.

The model including qualitative information significantly dominates the hard facts model in virtually all respects. The inspection of ROC indicates that, contrary to credit analysts' intuition, the hard-facts variable based on financial ratios performs better in the high-risk region than the softfacts variable based on credit analysts' judgments. It is not possible to infer from the results of this study whether the increase in performance of a credit rating model justifies the additional costs of obtaining qualitative information for a particular bank. Yet, this study shows that subjective judgments are indeed capable of yielding valuable information and improve credit rating systems which are based solely on quantitative information by considerable amounts. It uses several methods to quantify this increase in forecast quality and is based, as one of the first studies so far, on an extensive set of "real world" credit data.

JEL Classification: G21, C25, C10

Suggested Citation

Lehmann, Bina, Is it Worth the While? The Relevance of Qualitative Information in Credit Rating (April 17, 2003). EFMA 2003 Helinski Meetings. Available at SSRN: https://ssrn.com/abstract=410186 or http://dx.doi.org/10.2139/ssrn.410186

Bina Lehmann (Contact Author)

Universitaet Konstanz - Center of Finance & Econometrics (CoFE) ( email )

Konstanz, 78457
Germany

Register to save articles to
your library

Register

Paper statistics

Downloads
2,014
rank
6,946
Abstract Views
6,950
PlumX Metrics