Medical Malpractice and Black-Box Medicine

Abstract

The explosive proliferation of health data has combined with the rapid development of machine-learning algorithms to enable a new form of medicine: black-box medicine. In this phenomenon, algorithms troll through tremendous databases of health data to find patterns that can be used to guide care, whether by predicting unknown patient risks, selecting the right drug, suggesting a new use of an old drug, or triaging patients to preserve health resources. These decisions differ in kind from previous data-based decisions because black-box medicine is, by its nature, opaque; that is, the bases for black-box decisions are unknown and unknowable. Black-box medicine raises a number of legal questions, ranging from how to shape incentives for its development to how to regulate its growth and quality. One key question is how black-box medicine will influence the medical malpractice liability of health-care providers. How should tort liability apply to providers who cannot know the mechanistic underpinnings of the treatment they recommend? Must they learn as much as they can about the way algorithms are developed and verified? Or can they rely on the assurances of the developer without more knowledge? And how can they obtain the informed consent of patients?

This chapter explores the medical malpractice implications of black-box medicine. It briefly introduces the phenomenon and then considers how the tort system does, can, and should regulate the behavior of providers and health-care facilities using black-box medical techniques. It concludes that while providers and facilities are ill-suited to evaluate the substantive accuracy of black-box medical algorithms, they can and should be required to exercise due care to evaluate procedural quality — the expertise of the developer and the availability of independent external validation — when implementing black-box algorithms in a health-care facility or using them to care for patients.

Keywords: black-box medicine, precision medicine, medical malpractice, medical algorithms