Controversial Classifiers — the Perils of Prediction Algorithms in Public Administration

Abstract

The application to people of prediction algorithms, especially ones based on statistical classifier methods, can be perilous — and not only when limitations, bias and systemic effects are ill-understood by those using them as has often been the case. This is topical as such methods, sometimes labelled artificial intelligence, machine learning algorithms, automated decision making, predictive analytics, data analytics, or similar, have been promoted and to some extent adopted by government agencies. They have been used in the field of social services, criminal justice, policing, benefits fraud, and others.

Evidence is presented of the many risks and problems that have emerged with such uses. Unlawfulness, breach of the principles of good public administration such as fairness, transparency and giving explanations, or simply a failure to work in any meaningful sense, have led to cancellation or criticism of many systems. Scandals and real, serious harms to people have occurred.

However, the article makes the case that even if it worked lawfully and soundly, the singular, isolated, and indiscriminate use of such a system by a public authority is fundamentally unjust, being a process of automated, decontextualised classification that should have no place in the governance of a democratic society. Judicial and administrative actions need to be based on demonstrable evidence, not obscure statistical predictions. Going further, it could be considered a means of establishing an otherness that is creating social boundaries and may be misused for political purposes. This potentially undermines democracy.

The article concludes that any algorithmic process used by public authorities to predict attributes or circumstances of people should not happen in isolation nor be a single factor for decision-making. In any situation, such methods should be deployed with great care, expert assistance, and democratic oversight and control — if at all.

Keywords: Algorithms, Classification, Statistical Classifiers, Predictive Analytics, Data Analytics, Artificial Intelligence, AI, Machine Learning, Automated Decision Making, Government Services, Public Sector, Public Administration, Ethics, Trust, Law

JEL Classification: C38, H11, H83, Z13, Z18