Limitations of the 'Four-Fifths Rule' and Statistical Parity Tests for Measuring Fairness

8 GEO. L. TECH. REV. ___ (2023)

45 Pages Posted: 12 Dec 2023

See all articles by Manish Raghavan

Manish Raghavan

Massachusetts Institute of Technology (MIT)

Pauline Kim

Washington University in St. Louis - School of Law

Date Written: November 6, 2023

Abstract

To ensure the fairness of algorithmic decision systems, such as employment selection tools, computer scientists and practitioners often refer to the so-called “four-fifths rule” as a measure of a tool’s compliance with anti-discrimination law. This reliance is problematic because the “rule” is in fact not a legal rule for establishing discrimination, and it offers a crude test that will often be over- and under-inclusive in identifying practices that warrant further scrutiny. The “four-fifths rule” is one of a broader class of statistical tests, which we call Statistical Parity Tests (SPTs), that compare selection rates across demographic groups. While some SPTs are more statistically robust, all share some critical limitations in identifying disparate impacts retrospectively. When these tests are used prospectively as an optimization objective shaping model development, additional concerns arise about the development process, behavioral incentives, and gameability. In this article, we discuss the appropriate role for SPTs in algorithmic governance. We suggest a combination of measures that take advantage of the additional information present during prospective optimization, providing greater insight into fairness considerations when building and auditing models.

Keywords: four-fifths rule, algorithmic discrimination

Suggested Citation

Raghavan, Manish and Kim, Pauline, Limitations of the 'Four-Fifths Rule' and Statistical Parity Tests for Measuring Fairness (November 6, 2023). 8 GEO. L. TECH. REV. ___ (2023), Available at SSRN: https://ssrn.com/abstract=4624571

Manish Raghavan (Contact Author)

Massachusetts Institute of Technology (MIT) ( email )

77 Massachusetts Avenue
50 Memorial Drive
Cambridge, MA 02139-4307
United States

Pauline Kim

Washington University in St. Louis - School of Law ( email )

Campus Box 1120
St. Louis, MO 63130
United States
314-935-8570 (Phone)
314-935-5356 (Fax)

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
220
Abstract Views
816
Rank
286,239
PlumX Metrics