Regulating Black-Box Medicine

Abstract

Data drive modern medicine. And our tools to analyze those data are growing ever more powerful. As health data are collected in greater and greater amounts, sophisticated algorithms based on those data can drive medical innovation, improve the process of care, and increase efficiency. Those algorithms, however, vary widely in quality. Some are accurate and powerful, while others may be riddled with errors or based on faulty science. When an opaque algorithm recommends an insulin dose to a diabetic patient, how do we know that dose is correct? Patients, providers, and insurers face substantial difficulties in identifying high-quality algorithms; they lack both expertise and proprietary information. How should we ensure that medical algorithms are safe and effective?

Medical algorithms need regulatory oversight, but that oversight must be appropriately tailored. Unfortunately, FDA has suggested that it will regulate algorithms under its traditional framework; this relatively rigid system is likely to stifle innovation and to block the development of more flexible, current algorithms.

This Article draws upon ideas from the New Governance movement to suggest a different approach. FDA should pursue a more adaptive regulatory approach with requirements that developers disclose information underlying their algorithms. Disclosure would allow FDA oversight to be supplemented by evaluation by providers, hospitals, and insurers. This collaborative approach would supplement the agency’s review with ongoing real-world feedback from sophisticated market actors. Medical algorithms have tremendous potential, but ensuring that such potential is developed in high-quality ways demands a careful balancing between public and private oversight, and a role for FDA that mediates—but does not dominate—the rapidly developing industry.

Keywords: black-box medicine, algorithms, regulation, innovation