Precision-Recall Curves

Abstract

ROC curves and the area under them, AUC-ROC, are sometimes used to assess model fit in international relations and other research. At the same time, the outcomes of interest like civil war or interstate war onset are rare events, leading to data that mostly consists of 0’s, with few 1’s. AUC-ROC is misleading for such data and overstates the actual performance of a model because it does not capture what likely is low precision in the predictions–many false positives for every true positive. Precision-recall curves and the area under them, AUC-PR, are based on precision rather than the false positive rate, and thus better reflect model performance when predicting rare outcomes.

Keywords: precision, recall, rare events, prediction