Trustworthy Artificial Intelligence in Education: Pitfalls and Pathways

Abstract

Across the world, Artificial Intelligence (AI) applications are entering all domains of our lives, including the educational environment. This development was further spurred by the COVID-19 pandemic, which rendered many educational institutions dependent on (AI-enabled) digital learning tools to continue their activities. While the use of AI systems can lead to numerous benefits, it can also entail ethical risks, which are increasingly appearing on legislators’ agendas. Many of these risks are context-specific and increase in significance when vulnerable individuals are involved, asymmetries of power exist, or human rights and democratic values are at stake more generally. Surprisingly, however, regulators have thus far paid only little attention to the specific risks arising in the context of Artificial Intelligence in education (AIED). This paper hence aims to assess the ethical challenges posed by AIED. Its normative framework consists of the seven requirements for Trustworthy AI, as set out in the Ethics Guidelines of the European Commission’s High-Level Expert Group on AI. After an overview of the broader context in which these requirements took shape (Section 2), the paper examines each requirement in the educational realm, as well as the pitfalls that should be addressed to enable their realisation (Section 3). Particular attention is given to the special role of education in shaping people’s minds, and the manner in which this role can be used both to empower and exploit individuals. The paper notes that AIED’s main strengths – offering education on a wider scale, through more flexible and individualised learning methods, and through the closer monitoring of students’ reception of the materials – are also its main liabilities when left unchecked. Finally, the paper discusses various pathways that policymakers should consider to foster Trustworthy AIED beyond the adoption of guidelines (Section 4), before concluding (Section 5).