Confidence Bands for Roc Curves

10 Pages Posted: 13 Oct 2008

See all articles by Sofus Macskassy

Sofus Macskassy

Fetch Technologies, Inc

Foster Provost

New York University

Michael L. Littman

affiliation not provided to SSRN

Multiple version iconThere are 2 versions of this paper

Date Written: 2003


We address the problem of comparing the performance of classifiers. In this paper we study techniques for generating and evaluating bands on ROC curves. Historically this has been done using one-dimensional confidence intervals by freezing one variable - false-positiverate, or threshold on the classification scoring function. Weadapt two prior methods and introduce a new radial sweepmethod to generate confidence bands. We show, throughempirical studies, that the bands are too tight and introducea general optimization methodology for creatingbands that better fit the data, as well as methods for evaluatingconfidence bands. We show empirically that theoptimized confidence bands fit much better and that, usingour new evaluation method, it is possible to gauge therelative fit of different confidence bands.

Suggested Citation

Macskassy, Sofus and Provost, Foster and Littman, Michael L., Confidence Bands for Roc Curves (2003). NYU Working Paper No. 2451/14153, Available at SSRN:

Sofus Macskassy (Contact Author)

Fetch Technologies, Inc ( email )

2041 Rosecrans Ave
Suite 245
El Segundo, CA 90245
United States


Foster Provost

New York University ( email )

44 West Fourth Street
New York, NY 10012
United States

Michael L. Littman

affiliation not provided to SSRN

No Address Available

Here is the Coronavirus
related research on SSRN

Paper statistics

Abstract Views
PlumX Metrics