When to Consult Precision-Recall Curves

16 Pages Posted: 3 Apr 2019 Last revised: 13 Jul 2019

See all articles by Jonathan A Cook

Jonathan A Cook

Public Company Accounting Oversight Board

Vikram Ramadas

Public Company Accounting Oversight Board

Date Written: June 1, 2019

Abstract

ROC curves are commonly used to evaluate predictions of binary outcomes. When there are a small percentage of items of interest (as would be the case with fraud detection, for example), ROC curves can provide an inflated view of performance. This can cause challenges in trying to determine which set of predictions is better. This article discusses the conditions under which precision-recall curves may be preferable to ROC curves. As an illustrative example, we compare two commonly used fraud predictors (Beneish's (1999) M-score and Dechow et al.'s (2011) F-score) using both ROC and precision-recall curves.

To aid the reader with using precision-recall curves, this article also introduces a Stata module to plot them. This module is now available on the Boston College Statistical Software Components (SSC) archive and can be installed by typing ssc install prtab in Stata.

Keywords: Precision-recall curves, Classifier evaluation, ROC curves, Fraud detection

Suggested Citation

Cook, Jonathan A and Ramadas, Vikram, When to Consult Precision-Recall Curves (June 1, 2019). Available at SSRN: https://ssrn.com/abstract=3350582 or http://dx.doi.org/10.2139/ssrn.3350582

Jonathan A Cook (Contact Author)

Public Company Accounting Oversight Board ( email )

1666 K Street, NW
Washington, DC 20006-2
United States

Vikram Ramadas

Public Company Accounting Oversight Board ( email )

1666 K Street, NW
Washington, DC 20006-2
United States

Here is the Coronavirus
related research on SSRN

Paper statistics

Downloads
160
Abstract Views
1,230
rank
202,185
PlumX Metrics