Constructing 'Proper' ROCs from Ordinal Response Data Using Weighted Power Functions
Medical Decision Making, Forthcoming
31 Pages Posted: 2 Dec 2012 Last revised: 18 Aug 2013
Date Written: April 14, 2013
Abstract
Background: Receiver operating characteristic (ROC) analysis is the standard method for describing the accuracy of diagnostic systems where the decision task involves distinguishing between two mutually exclusive possibilities. The popular binormal curve-fitting model usually produces ROCs that are improper in that they do not have the ever-decreasing slope required by signal detection theory. Not infrequently, binormal ROCs have visible hooks that falsely imply worse-than-chance diagnostic differentiation where the curve lies below the no-information diagonal.
Objective: To present and evaluate a two-parameter, weighted power function (WPF) model that always results in a proper ROC curve with a positive, monotonically decreasing slope.
Design/Method: Computer simulation study comparing results from binormal and WPF models.
Results: The WPF model produces ROC curves that are less biased and closer to the true values than curves obtained using the binormal model. The better performance of the WPF model follows from its design constraint as a necessarily proper ROC.
Conclusions: The WPF model fits a broader variety of data sets than previously published power function models while maintaining straightforward relationships among the original decision variable, specific operating points, ROC curve contours, and model parameters. Compared to other proper ROC models, the WPF model is distinctive in its simplicity, and it avoids the flaws of the conventional binormal ROC model.
Keywords: receiver operating characteristic, ROC, binormal ROC, proper ROC
JEL Classification: C13, C15
Suggested Citation: Suggested Citation