Understanding Discrimination in the Scored Society
38 Pages Posted: 17 Jan 2015
Date Written: January 15, 2015
Modern society features an abundance of situations in which complicated decisions are reached automatically and while relying on the troves of Big Data at the decider's disposal. Furthermore, many of these cases feature a scoring dynamic; a situation in a small yet powerful group of individuals secretly structure a scoring scheme and use it to treat similar individuals differently. The seemingly arbitrary scoring process also unfolds in a manner which is incomprehensible to those affected by it.
Scoring has been carried out for years in the realm of consumer credit. Yet the age of big data is leading to the dissemination of these practices to many other contexts. While promoting efficiency and generating important knowledge, the accelerating use of scoring brings about a variety of problems. In a recent important article, "The Scored Society," Professors Danielle Citron and Frank Pasquale examine these issues, explain their severity and offer an innovative set of responses. This Article continues Citron's and Pasquale's analytical journey, and focuses on an important concern surfacing in discussions and analyses of the scored society and the troubles big data analytics bring about -- is that of discrimination.
The Article therefore draws out several antidiscrimination paradigms which on their face pertain to the dynamics discussed in The Scored Society, and big data in general. Such analysis allows for recognizing which discrimination-based concerns are especially acute in the scored society, as well as setting forth initial proposed responses for mitigating them, when possible. The Article proceeds as follows: after a brief Introduction mapping the confines of the debate, the Article moves to Part I, where it generally addresses the notion of "discrimination" and its relevance to the issue at hand. Part II -- the heart of this Article -- identifies the discrimination-based concerns which relate to the mistreatment of "protected groups." There, the Article demonstrates the possible concerns while relying on race as a key example of a "protected group" and distinguishing between explicit discrimination, implicit discrimination, and instances of disparate impact. In Part III, the Article takes a brief look at selected discrimination concerns which go beyond protected groups. It generally finds these latter problems relatively easy to resolve. Finally, in the Conclusion, the Article argues that even though the scoring process is seemingly ridden with discrimination-based concerns, it certainly should not be categorically abandoned, as it might even promote antidiscrimination objectives when carried out properly.
Keywords: Big data, data mining, profiling, credit rating, discrimination, racial discrimination, implicit discrimination, disparate impact, automatic decisions, privacy.
Suggested Citation: Suggested Citation