AI as a Medical Device: Between the Medical Devices Framework and the General AI Regulation
Time to Reshape the Digital Society, 40th Anniversary of the CRIDS, conference book, University of Namur, Belgium, November 2021
11 Pages Posted: 19 Sep 2022
Date Written: September 28, 2021
Abstract
This paper is the follow-up to my previous paper ‘AI as a Medical Device: Is It Enough to Ensure External Performance Transparency and Accountability?’ published in March 2020. Back then, the general approach to regulate artificial intelligence (‘AI’) was in its development stage. The analysis of AI used in healthcare was thus based on the Medical Devices Framework (‘MDF’) and identified three main issues of the framework:
- limitations in the regulated subjects (not enough considerations for the roles of users),
- limitations in the regulatory scope (focus on safety and performance rather than on transparency and accountability);
- limitations in procedures (self-learning nature of AI and data dependence are not sufficiently covered).
This follow-up article analyses how the EC Proposal for the AI Act issued in April 2021 deals with the three identified limitations. I show that the EU legislator suggested rather good solution for the synergy of the future AI Act and the contextualized legal framework for the use of AI in healthcare (MDF). I explain how the issued I identified before are solved in the proposed act.
I also describe the new issues non-resolved in the proposed AI Act. I suggest taking into consideration the situations where AI is used in multi-stakeholders’ environment (which is the majority of AI’s usage) and better distinguish the rights and obligations of specialists using AI systems and organizations they work for (in medical contexts it is physicians and medical organizations respectively). In addition, the role of AI’s beneficiaries, persons in relation to whom AI produces decisions (in healthcare it is usually patients) shall be clarified. Another issue to be solved in the proposed AI Act is the guidance on how transparency and interpretability shall be understood. This would clarify if the black-box AI models are considered to be compatible with the interpretability requirement and what kind of methods are eligible for applying by AI providers to comply with the said requirement.
Keywords: AI, medical devices, transparency, accountability, AI Act, artificial intelligence, machine learning, black-box, healthcare, safety, performance, informed consent, high-risk AI
Suggested Citation: Suggested Citation