A Review of the ICO’s Draft Guidance on the AI Auditing Framework
15 Pages Posted: 25 Jun 2020
Date Written: May 12, 2020
The Information Commissioner's Office (ICO) timely call for consultation regarding their ‘Guidance on the AI auditing framework: Draft guidance for consultation’ (February 2020) is part of a growing literature concerning the governance of Artificial Intelligence (AI) systems. The ICO’s draft leads the national conversation by producing guidance that encompasses both technical (ex. system impact assessments) and non-technical (ex. human oversight) components to governance and represents a significant milestone in the movement towards standardising AI governance. Welcoming this crucial intervention, we summarise and critically evaluated each section of the draft guidance, offering feed-back in line with the call for consultation. We conclude with a note on what we anticipate will be future debates and by presenting our general recommendations. These are namely:
Data and AI: explicit discussion of the relationship between data protection and how it translates into auditing of AI systems.
Case studies: templates development produced through discussion in an open forum for relevant stakeholders.
Risks of other Machine Learning (ML) systems: in addition to guidance on Supervised Learning further guidance would be welcome about risks presented in other forms of ML.
Regression and Forecasting: discuss the metrics and methods used when an AI system is used to tackle a Regression or a Forecasting problem.
Target audience: better specification (Data Scientists, DPOs, etc.) or a framework should be created where each group is targeted within a structure that integrates their respective duties.
Suggested Citation: Suggested Citation