Confidence Bands for Roc Curves

10 Pages Posted: 13 Oct 2008

See all articles by Sofus Macskassy

Sofus Macskassy

Fetch Technologies, Inc

Foster Provost

New York University

Michael L. Littman

affiliation not provided to SSRN

Multiple version iconThere are 2 versions of this paper

Date Written: 2003

Abstract

We address the problem of comparing the performance of classifiers. In this paper we study techniques for generating and evaluating bands on ROC curves. Historically this has been done using one-dimensional confidence intervals by freezing one variable - false-positiverate, or threshold on the classification scoring function. Weadapt two prior methods and introduce a new radial sweepmethod to generate confidence bands. We show, throughempirical studies, that the bands are too tight and introducea general optimization methodology for creatingbands that better fit the data, as well as methods for evaluatingconfidence bands. We show empirically that theoptimized confidence bands fit much better and that, usingour new evaluation method, it is possible to gauge therelative fit of different confidence bands.

Suggested Citation

Macskassy, Sofus and Provost, Foster and Littman, Michael L., Confidence Bands for Roc Curves (2003). Information Systems Working Papers Series, Vol. , pp. -, 2003. Available at SSRN: https://ssrn.com/abstract=1282995

Sofus Macskassy (Contact Author)

Fetch Technologies, Inc ( email )

2041 Rosecrans Ave
Suite 245
El Segundo, CA 90245
United States

HOME PAGE: http://www.fetch.com

Foster Provost

New York University ( email )

44 West Fourth Street
New York, NY 10012
United States

Michael L. Littman

affiliation not provided to SSRN

No Address Available

Register to save articles to
your library

Register

Paper statistics

Downloads
49
Abstract Views
546
PlumX Metrics