Legal Singularity and the Reflexivity of Law

Abstract

This paper argues that those pursuing legal AI – particularly those interested in the so-called ‘legal singularity’ – misunderstand the nature of both the law and the technology, with the effect that not only would they fail to solve the very real problems with the law, but they could potentially both make them worse and cause new ones. Drawing on insights and concepts from multiple disciplines, my argument rests on the idea that law functions as a reflexive societal institution. As such, I argue, the law cannot be neutral, but is instead contextual and contingent on the circumstances of the time, imbued with normative assumptions and priorities, and reifies the interests and goals of its writers, practitioners, and adjudicators. This reflexive functioning should be understood as distinct from the role that law plays in society as a result of its reflexivity. Functioning as a reflexive societal institution, I argue, the law’s role has not only been to reflect the inequalities and injustices in society, but to repeat, reinforce, and re-encode them back into society. As such, just as it matters what goals and priorities are being pursued in the design, deployment, and use of algorithmic systems, so too it matters what goals and priorities those working within the law are pursuing.

I develop my argument in two parts. In the first, I argue that algorithmic systems are not – and may never be – capable of replacing humans in the law. I emphasise, though, that critiques of legal AI that focus on the technical limitations of algorithmic systems, while important, do not get to the heart of the structural questions in which I am primarily interested. In the second part of my argument, therefore, I develop a critique of the power relations and structural effects of legal AI as it is commonly envisaged. Adopting Foucauldian theories of governmentality and again drawing from multiple disciplines, I begin by locating the ideological underpinnings and motivations of legal AI as being part of a process of neoliberal rationalisation; replacing the qualitative, normative values of law with supposedly rational, objective, quantitative metrics and logics based on statistical and economic thinking. As part of this process of rationalisation, the law is often problematised by legal AI proponents as slow, costly, inefficient, complex, unpredictable, in need of optimisation, and thus amenable to techno-solutionist interventions. As a result, in my view, by framing the case for legal AI in terms that are fundamentally concerned with the quality of the law’s functioning, legal AI proponents not only fail to consider the nature of the law’s role in society but prioritise the kind of market-oriented and commercially-driven ways of thinking that contribute to the development of problems with that role in the first place. Without a critical examination of the law’s role in society, I argue, legal AI proponents therefore risk developing systems that will primarily make law ‘better’ at extending and reinforcing hierarchies, maintaining the law’s exclusionary effects, and reifying the dominance and power of capital.

Keywords: Law, artificial intelligence, legal tech, legal AI, governmentality