Measuring the Performance of Corporate Bond Ratings
Special Comment, April 2003
28 Pages Posted: 25 Jun 2007
A rating system's performance should be measured by its success or failure in meeting its objectives. This Special Comment discusses various metrics which may be used to judge rating systems and the application of these metrics to Moody's historical performance. Moody's corporate bond ratings are intended to be "accurate" and "stable" measures of relative credit risk, as determined by each issuer's relative fundamental creditworthiness and without reference to explicit time horizons. Moody's performance should therefore be measured by both rating accuracy (the correlation between ratings and defaults) and rating stability (the frequency and magnitude of ratings changes). To measure the accuracy of its corporate bond ratings Moody's tracks the cumulative accuracy profiles and accuracy ratios of its ratings, since relative rating accuracy is a primary objective of its rating system. Moody's also tracks investment-grade default rates and the average rating of defaulting issuers prior to their defaults. These metrics measure Moody's success at meeting a secondary cardinal or absolute rating system objective, namely that ratings be useful to investors who employ simple rating "cutoffs" in their investment eligibility guidelines. Moody's does not target specific default rates for the individual rating categories. For rating systems that do have this type of cardinal objective, performance can be measured by the stability of realized default rates by rating category around target default rates. To measure ratings stability Moody's tracks the frequency of rating changes, the frequency of large rating changes, and the frequency of rating reversals. Moody's ratings should change only when relative fundamental creditworthiness changes. Since relative fundamental credit risk generally changes quite slowly, ratings should be stable, especially when compared with other market based risk measures. Infrequent reversals, and stability more generally, facilitate the use of ratings in connection with investment eligibility guidelines and performance benchmarks. The accuracy and stability of Moody's ratings has fluctuated over time. Moody's rating accuracy ratios are currently within historical norms, which range from the lows of the early 1980's to a high in the mid-1990's. Current rating stability is low by historical norms. Changes in ratings performance may reflect either changes in the quality of the ratings process or changes in the environment that make defaults more or less difficult to predict, or make fundamental creditworthiness more volatile, thus making rating stability harder to maintain. To control for changes in the economic environment, we also compare the accuracy and stability of Moody's ratings to the accuracy and stability of other credit risk measures, such as bond market-implied ratings inferred from observed credit spreads. Over the past four years, compared to Moody's ratings, bond market-implied ratings have been more correlated with defaults at one-year horizons, but their accuracy ratios are comparable as the horizon lengthens to three years. Moody's ratings have been much more stable than market-based credit opinions. In any given twelve-month period, there is about one rating action for every four issuers and only one out of every 100 issuers experiences a rating reversal (a rating change followed by a rating change in the opposite direction). In contrast, based on market-implied ratings (inferred from monthly bond spreads), nine out of ten issuers experience rating changes each year and seven out of ten experience rating reversals. This study is limited to the assessment of the accuracy and stability of Moody's ratings; however, a complete analysis of Moody's rating system also requires an analysis of its outlooks and Watchlist designations, which will be a subject of Moody's future research.
Keywords: credit ratings, performance, default
Suggested Citation: Suggested Citation