How AI Challenges the Medical Device Regulation: Patient Safety, Benefits, and Intended Uses

Daria Onitiu, Sandra Wachter, Brent Mittelstadt, How AI challenges the medical device regulation: patient safety, benefits, and intended uses, Journal of Law and the Biosciences (9 April 2024).

50 Pages Posted: 5 Dec 2023 Last revised: 11 Apr 2024

See all articles by Daria Onitiu

Daria Onitiu

University of Oxford - Oxford Internet Institute

Sandra Wachter

University of Oxford - Oxford Internet Institute

Brent Mittelstadt

University of Oxford - Oxford Internet Institute

Date Written: July 20, 2023

Abstract

This article examines whether the EU Medical Device Regulation (MDR) adequately addresses novel risks of AI-based medical devices (AIaMD), focusing on AI medical imaging tools. It examines two questions: first, does the MDR effectively deal with issues of adaptability, autonomy, bias, opacity and need of trustworthiness of AIaMD? Second, does the manufacturer’s translation of the MDR’s requirements close a discrepancy between an AIaMDs’ expected benefit and actual clinical utility of assessing device safety and effectiveness beyond the narrow performance of algorithms? While the first question has previously received attention in scholarly literature on regulatory and policy tensions on AIaMD generally, and work on future technical standard-setting, the second has been comparatively overlooked. We argue that effective regulation of AIaMD requires framing notions of patient safety and benefit within the manufacturer’s articulation of the device’s intended use, as well as reconciling tensions. These tensions are on (i) patient safety and knowledge gaps surrounding fairness, (ii) trustworthiness and device effectiveness, (iii) the assessment of clinical performance, and (iv) performance updates. Future guidance needs to focus on the importance of translated benefits, including nuanced risk-framing and looking at how the limitations of AIaMD inform the intended purpose statement in the MDR.

Note:
Funding declaration: This work has been supported through research funding provided by the Wellcome Trust (grant nr 223765/Z/21/Z), Sloan Foundation (grant nr G-2021-16779), Department of Health and Social Care, and Luminate Group. Their funding supports the Trustworthiness Auditing for AI project and Governance of Emerging Technologies research programme at the Oxford Internet Institute, University of Oxford.

Conflict of Interests: We confirm that no potential conflict of interest was reported by the authors.

Keywords: Artificial Intelligence, Medical Device Regulation, AI-based medical device

Suggested Citation

Onitiu, Daria and Wachter, Sandra and Mittelstadt, Brent, How AI Challenges the Medical Device Regulation: Patient Safety, Benefits, and Intended Uses (July 20, 2023). Daria Onitiu, Sandra Wachter, Brent Mittelstadt, How AI challenges the medical device regulation: patient safety, benefits, and intended uses, Journal of Law and the Biosciences (9 April 2024). , Available at SSRN: https://ssrn.com/abstract=4638548 or http://dx.doi.org/10.2139/ssrn.4638548

Daria Onitiu (Contact Author)

University of Oxford - Oxford Internet Institute ( email )

1 St. Giles
University of Oxford
Oxford OX1 3PG Oxfordshire, Oxfordshire OX1 3JS
United Kingdom

Sandra Wachter

University of Oxford - Oxford Internet Institute ( email )

1 St. Giles
University of Oxford
Oxford OX1 3PG Oxfordshire, Oxfordshire OX1 3JS
United Kingdom

Brent Mittelstadt

University of Oxford - Oxford Internet Institute ( email )

1 St. Giles
University of Oxford
Oxford OX1 3PG Oxfordshire, Oxfordshire OX1 3JS
United Kingdom

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
227
Abstract Views
992
Rank
276,208
PlumX Metrics