Model Rules on Impact Assessment of Algorithmic Decision-Making Systems used by Public Administration, Report of the European Law Institute

Abstract

Public bodies have made decisions and rules from time immemorial. The nature of the particular public bodies perforce varies as between legal systems. It is, in addition, commonplace to accept that decisions may also be subject to public law in certain instances when they have been made by private actors. There is nonetheless a foundational commonality underlying the preceding heterogeneity, which is that while the institutions might have varied in certain respects as between legal systems, the decisions were made by human beings. There was an individual, or institution, that made the contested rule or decision. The subject matter of these Model Rules attests to an important change in this regard, since the reality is that in many instances it is not possible to trace a decision back to a discrete individual. The operative decision may be made by an algorithm, or some other form of automated decision-making. Humans may still be involved in such decisional processes, in the sense that they may design the algorithm, and there may also be some human involvement before the operative decision is taken. However, a system may be fully automated when set up, such that the output/decision can occur without human involvement, and some systems make provision for the algorithm to learn and develop. It is, therefore, unsurprising that the existence of such automated systems broadly conceived poses novel problems for both public and private law. This is attested to by the plethora of initiatives dealing with such matters emanating from bodies such as the European Union (EU) and the Council of Europe, as well as from particular nation States.

It is important to underline that these Model Rules have been designed so that they are not dependent on EU law and can be implemented in non-EU legal systems. In other words, they must be able to fit into different legal contexts that do not incorporate fundamental elements of EU data protection law such as the General Data Protection Regulation (GDPR). However, the rules presented have been developed in such a way as to ensure that they are compatible not only with existing EU law, but also with the law currently being drafted, in particular the Draft Regulation on Artificial Intelligence (AI). The latter draft concerns to a large extent the AI projects developed or used by administrations. The Model Rules complement the Draft Regulation’s approach by providing specific safeguards for democracy, the right to good administration and the rule of law when algorithmic decision-making systems are used by the public administration. This is exemplified by the provision made for case-specific impact assessments including public and expert participation.

There are various ways in which concerns raised by algorithmic decision-making can be addressed. The central idea underlying these Model Rules is for an Impact Assessment to be conducted. The very variety of situations in which algorithmic decision-making is employed precludes a one-size fits all approach. This would lead to rules that were too rigorous for some such systems, and too generous for others. The approach in the Model Rules is therefore variegated. A legal system that adopts the Model Rules can specify that certain such systems fall into Annex 1, which denotes that they are high risk, and hence always subject to an Impact Assessment. It can, to like effect, stipulate that other systems should fall into Annex 2, because they are regarded as low risk and therefore do not warrant such an Assessment. There may, however, be other such systems that cannot readily be classified ex ante as falling within either Annex 1 or Annex 2. These systems are subject to an initial risk evaluation in accordance with a screening procedure. An Impact Assessment is then required if the system constitutes at least a substantial risk according to the screening procedure. The Model Rules set out in detail the nature of the Impact Assessment, which is framed by a prior scoping procedure designed to target the more particular issues on which such an Assessment should focus. The Impact Assessment is intended to be both measured, in the sense of addressing the benefits as well as the risks from the use of algorithmic decision-making; and proportionate, in the sense of not being too burdensome for the authorities that have to conduct them. There are, however, further requirements for such systems that are regarded as high risk, either because they fall within Annex 1, or because they are deemed to be so as a result of a particular Impact Assessment. The further requirements are set out in Chapter 3 of the Model Rules and entail, inter alia, scrutiny of the Impact Assessment by an expert board, as well as the opportunity for public participation. The Model Rules also make provision for a Supervisory Authority to oversee the preceding processes, and specify the circumstances in which a legal challenge might be made.

Keywords: impact assessment, model rules, algorithmic decision-making