How AI Challenges the Medical Device Regulation: Patient Safety, Benefits, and Intended Uses
Daria Onitiu, Sandra Wachter, Brent Mittelstadt, How AI challenges the medical device regulation: patient safety, benefits, and intended uses, Journal of Law and the Biosciences (9 April 2024).
50 Pages Posted: 5 Dec 2023 Last revised: 11 Apr 2024
Date Written: July 20, 2023
Abstract
This article examines whether the EU Medical Device Regulation (MDR) adequately addresses novel risks of AI-based medical devices (AIaMD), focusing on AI medical imaging tools. It examines two questions: first, does the MDR effectively deal with issues of adaptability, autonomy, bias, opacity and need of trustworthiness of AIaMD? Second, does the manufacturer’s translation of the MDR’s requirements close a discrepancy between an AIaMDs’ expected benefit and actual clinical utility of assessing device safety and effectiveness beyond the narrow performance of algorithms? While the first question has previously received attention in scholarly literature on regulatory and policy tensions on AIaMD generally, and work on future technical standard-setting, the second has been comparatively overlooked. We argue that effective regulation of AIaMD requires framing notions of patient safety and benefit within the manufacturer’s articulation of the device’s intended use, as well as reconciling tensions. These tensions are on (i) patient safety and knowledge gaps surrounding fairness, (ii) trustworthiness and device effectiveness, (iii) the assessment of clinical performance, and (iv) performance updates. Future guidance needs to focus on the importance of translated benefits, including nuanced risk-framing and looking at how the limitations of AIaMD inform the intended purpose statement in the MDR.
Note:
Funding declaration: This work has been supported through research funding provided by the Wellcome Trust (grant nr 223765/Z/21/Z), Sloan Foundation (grant nr G-2021-16779), Department of Health and Social Care, and Luminate Group. Their funding supports the Trustworthiness Auditing for AI project and Governance of Emerging Technologies research programme at the Oxford Internet Institute, University of Oxford.
Conflict of Interests: We confirm that no potential conflict of interest was reported by the authors.
Keywords: Artificial Intelligence, Medical Device Regulation, AI-based medical device
Suggested Citation: Suggested Citation