A generalised ROC curve

Abstract

A key point to assess the application of statistical learning models in Artificial Intelligence (AI) is the evaluation of their predictive accuracy. This because the "automatic" choice of an action crucially depends on the made prediction. While the best model in terms of fit to the observed data can be chosen using a "universal" - and therefore automatable - criterion, based on the models' likelihood, such as AIC and BIC, this is not the case for the best model in terms of predictive accuracy. When the variable to be predicted is continuous, the most employed criterion is the root mean squared error whereas, when the response variable is binary, the most employed criterion is the area under the ROC curve. We propose to generalise the ROC curve to obtain a predictive model selection criterion that does not depend on the nature of the response variable.

Keywords: Artificial Intelligence, Predictive Accuracy, Receiver Operating Characteristic curve

JEL Classification: C18, C40, C52