Critical Criteria for AI Impact Assessment: An Aggregated View

Abstract

This paper proposes a set of criteria for artificial intelligence (AI) impact assessment processes to facilitate public and private sector organizations to select the most appropriate impact assessment methodology for their specific contexts. To build this reference, the paper employs a dual research methodology: reviewing existing research on privacy impact assessment and examining recent discussions on potential elements and methods for AI-related impact assessment. We consolidate key findings from various impact assessment elements found in the literature and organize them in five dimensions: normative framework, process rules, methodology, engagement, and oversight. Within each of these dimensions, we propose a set of critical questions for meaningful impact assessment by integrating reflections on the challenges raised in the examined research, discussions on the ongoing impact assessment methodology, as well as investigations on impact assessment pitfalls. The resulting identified criteria aim at supporting AI governance in public and private organizations through offering a wider understanding of all the challenges observed in the literature. They are potentially useful to developers, regulators, and as a backbone of comparison for various standardization initiatives.

Keywords: Impact Assessment; AI Governance; Standards; Responsible AI