Deconstructing Design Decisions: Why Courts Must Interrogate Machine Learning and Other Technologies

Abstract

Technologies do not just come about. They are designed, and those design choices affect everything the technology touches. Yet unless a legal question directly implicates the technological design, courts are not likely to interrogate it. In this Article, we use examples from machine learning to demonstrate that the design choices matter even for cases where the legal questions do not involve technology directly. We start by describing formal “abstraction,” a fundamental design technique in computer science that treats systems and subsystems as defined entirely by their inputs, outputs, and the relationship that transforms inputs to outputs. We show how this technique causes the resulting technologies to be effectively making claims about responsibility and knowability that competes with courts’ own determinations. We further show that these claims are rendered invisible over time. Thus, we argue that courts must unearth—or deconstruct—the original design choices in order to understand the legal claims in a given case—even those cases that do not on their face appear to be about technological design.

There is, of course, a reasonable concern that courts are not capable or are not the best venue to make judgments about technological design. While we agree that courts are not the optimal front-line regulators of technology, we argue that they cannot avoid these questions as technologies begin show up in every type of case—a phenomenon that will only grow with time. But besides being forced to consider technology, courts are actually capable of doing so when motivated to. We demonstrate that in certain cases that clearly tee up technological design, such as products liability, copyright retransmission, and the functionality doctrines of intellectual property, courts have no problem diving in and questioning the design choices, asking what could have and should have been. Where courts can perform analysis in one arena, they can do so in another. Finally, through extended hypotheticals in the areas of negligence, discrimination, and criminal justice, we demonstrate how courts can effectively deconstruct technological design.

Keywords: Law & Technology, courts