How AI Unfairly Tilts the Playing Field: Privacy, Fairness, and Risk Shadows

37 Pages Posted: 26 Jul 2023

See all articles by Richard Warner

Richard Warner

Chicago-Kent College of Law

Robert H. Sloan

University of Illinois at Chicago

Date Written: July 22, 2023

Abstract

Private sector applications of artificial intelligence (AI) raise related questions of informational privacy and fairness. Fairness requires that market competition occurs on a level playing field, and uses of AI unfairly tilt the field. Informational privacy concerns arise because AI tilts the playing field by taking information about activities in one area of one’s life and using it in ways that impose novel risks in areas not formerly associated with such risks. The loss of control over that information constitutes a loss of informational privacy. To illustrate both the fairness and privacy issues, imagine, for example, that Sally declares bankruptcy after defaulting on $50,000 of credit card debt. She incurred the debt by paying for lifesaving medical treatment for her eight-year-old daughter. Post-bankruptcy Sally is a good credit risk. Her daughter has recovered, and her sole-proprietor business is seeing increased sales. Given her bankruptcy, however, an AI credit scoring system predicts that she is a poor risk and assigns her a low score. That low credit score casts a shadow that falls on her when her auto insurance company, which uses credit scores in its AI system as a measure of the propensity to take risks, raises her premium. Is it fair that saving her daughter’s life should carry with it the risk—realized in this case—of a higher premium? The pattern is not confined to credit ratings and insurance premiums. AI routinely creates risk shadows.

We address fairness questions in two steps. First, we turn to philosophical theories of fairness as equality of opportunity to spell out the content behind our metaphor of tilting the playing field. Second, we address the question of how, when confronted with a mathematically complex AI system, one can tell whether the system meets requirements of fairness. We answer by formulating three conditions whose violation makes a system presumptively unfair. The conditions provide a lens that reveals relevant features when policy makers and regulators investigate complex systems. Our goal is not to resolve fairness issues but to contribute to the creation of a forum in which legal regulators and affected parties can work to resolve them. The third of our three condition requires that systems incorporate contextual information about individual consumers, and we conclude by raising the question of whether our suggested approach to fairness significantly reduces informational privacy. We do not answer the question but emphasize that fairness and informational privacy questions can closely intertwine.

Suggested Citation

Warner, Richard and Sloan, Robert H., How AI Unfairly Tilts the Playing Field: Privacy, Fairness, and Risk Shadows (July 22, 2023). Available at SSRN: https://ssrn.com/abstract=4518510 or http://dx.doi.org/10.2139/ssrn.4518510

Richard Warner (Contact Author)

Chicago-Kent College of Law ( email )

565 West Adams St.
Chicago, IL 60661
United States

Robert H. Sloan

University of Illinois at Chicago ( email )

1200 W Harrison St
Chicago, IL 60607
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
105
Abstract Views
405
Rank
474,061
PlumX Metrics