Prediction Power as a Challenge for the Rule of Law

Abstract

This chapter unfolds the manifold implications of prediction power with a focus on the public sphere. It follows the thesis that questions of power are inherently questions of the rule of law, because the rule of law is understood as a dualistic principle serving a dual purpose: to both legitimise and constrain power. Originally aimed at limiting power towards the state, a contemporary understanding of the power-limiting function of the rule of law must consider different actors, multi-layered normative regimes and the dissolution of the public-private dichotomy. From the perspective of digital constitutionalism, Power is becoming less public at its core, and is nevertheless necessarily a public matter in democratic states and societies. Prediction Power is a specific variety of this development in digital constitutionalism: On the one hand, data-powerful private actors with their extensive capabilities to use predictive technologies pose a considerable challenge to the state's ability to limit power and regulate technological developments that pose risks to individual freedom and constitutional principles. On the other hand, state actors themselves possess predictive power and exercise it in various constellations. In these cases, the private sector's data power and the state's monopoly on the use of force are intertwined, as state actors utilise privately developed technologies. Consequently, the boundaries between the exercise of state power and private technologies are becoming increasingly blurred. The social impact on people, however, is clearly recognisable in fact but remains difficult to address legally. Predictions can lead to social sorting, discrimination, a lack of transparency, a restriction of legal protection, and the erosion of trust in public institutions. These developments affect all three dimensions of digital constitutionalism: the state, society, and the global sphere. They influence the changing role of states in digitalised societies, the societal impact of technological developments on social and legal norms, and the power of international companies over norms and regulations. Following an introduction, laying out what prediction power is and who its relevant actors are, the chapter describes the risks arising from prediction power which lead to a 'prediction society', the result of an epistemic shift in the generation and perception of knowledge. For this, the technical function, the connection between power and the rule of law, and the epistemic shift arising from predictions is critically analysed. This chapter argues that prediction power poses a fundamental challenge to the rule of law. This is due to the leveraging of collective data troves by the technologies that enable prediction power. Following the concept of 'new public analytics' put forward by Yeung, 1 the chapter lays out the connection between private technologies and state authorities in public analytics, discussing several use cases from public administration. Following this, the chapter analyses the implications of the AI Act on prediction power. Finally, the chapter concludes with a warning, urging further extension of intervention in the presence of risky data-driven technologies.

Keywords: AI, Data, Predictions, Power, Public Actors, Rule of Law