The Unfairness of Fair Machine Learning: Levelling down and strict egalitarianism by default

Abstract

In recent years fairness in machine learning (ML), artificial intelligence (AI), and algorithmic decision-making systems has emerged as a highly active area of research and development. To date, the majority of measures and methods to mitigate bias and improve fairness in algorithmic systems have been built in isolation from policy and civil societal contexts and lack serious engagement with philosophical, political, legal, and economic theories of equality and distributive justice. Most define fairness in simple terms, where fairness means reducing gaps in performance or outcomes between demographic groups while preserving as much of the accuracy of the original system as possible. This oversimplification of equality through fairness measures is troubling. Many current fairness measures suffer from both fairness and performance degradation, or “levelling down,” where fairness is achieved by making every group worse off, or by bringing better performing groups down to the level of the worst off. Levelling down is a symptom of the decision to measure fairness solely in terms of equality, or disparity between groups in performance and outcomes, while ignoring other relevant features of questions of distributive justice (e.g., welfare, priority) which are more difficult to quantify and measure. When fairness can only be measured in terms of distribution of performance or outcomes, corrective actions can likewise only target how these goods are distributed between groups: we refer to this trend as ‘strict egalitarianism by default’.

Strict egalitarianism by default runs counter to both the stated objectives of fairness measures as well as the presumptive aim of the field: to improve outcomes for historically disadvantaged or marginalised groups. When fairness can only be achieved by making everyone worse off in material or relational terms through injuries of stigma, loss of solidarity, unequal concern, and missed opportunities for substantive equality, something would appear to have gone wrong in translating the vague concept of ‘fairness’ into practice. Levelling down should be rejected in fairML because it (1) unnecessarily and arbitrarily harms advantaged groups in cases where performance is intrinsically valuable, such as medical applications of AI; (2) demonstrates a lack of equal concern for affected groups, undermines social solidarity, contributes to stigmatisation; (3) fails to live up to the substantive aims of equality law and fairML, and squanders the opportunity afforded by interest in algorithmic fairness to substantively address longstanding social inequalities; and (4) fails to meet the aims of many viable theories of distributive justice including pluralist egalitarian approaches, prioritarianism, sufficientarianism, and others.

This paper critically scrutinises these initial observations to determine how fairML can move beyond mere levelling down and strict egalitarianism by default. We examine the causes and prevalence of levelling down across fairML, and explore possible justifications and criticisms based on philosophical and legal theories of equality and distributive justice, as well as equality law jurisprudence. We find that fairML does not currently engage in the type of measurement, reporting, or analysis necessary to justify levelling down in practice. The types of decisions for which ML and AI are currently used, as well as inherent limitations on data collection and measurement, suggest levelling down can rarely be justified in practice. We propose a first step towards substantive equality in fairML: “levelling up” systems by design through enforcement of minimum acceptable harm thresholds, or “minimum rate constraints,” as fairness constraints. We likewise propose an alternative harms-based framework to counter the oversimplified egalitarian framing currently dominant in the field and push future discussion more towards substantive equality opportunities and away from strict egalitarianism by default.

Keywords: artificial intelligence, machine learning, fairness, bias, discrimination, distributive justice, philosophy, egalitarianism, equality law, anti-discrimination law