Rulemaking and Inscrutable Automated Decision Tools

36 Pages Posted: 9 Mar 2020

Date Written: 2019

Abstract

Complex machine learning models derived from personal data are increasingly used in making decisions important to peoples’ lives. These automated decision tools are controversial, in part because their operation is difficult for humans to grasp or explain. While scholars and policymakers have begun grappling with these explainability concerns, the debate has focused on explanations to decision subjects. This Essay argues that explainability has equally important normative and practical ramifications for decision-system design. Automated decision tools are particularly attractive when decisionmaking responsibility is delegated and distributed across multiple actors to handle large numbers of cases. Such decision systems depend on explanatory flows among those responsible for setting goals, developing decision criteria, and applying those criteria to particular cases. Inscrutable automated decision tools can disrupt all of these flows.

This Essay focuses on explanation’s role in decision-criteria development, which it analogizes to rulemaking. It analyzes whether, and how, decision tool inscrutability undermines the traditional functions of explanation in rulemaking. It concludes that providing information about the many aspects of decision tool design, function, and use that can be explained can perform many of those traditional functions. Nonetheless, the technical inscrutability of machine learning models has significant ramifications for some decision contexts. Decision tool inscrutability makes it harder, for example, to assess whether decision criteria will generalize to unusual cases or new situations and heightens communication and coordination barriers between data scientists and subject matter experts. The Essay concludes with some suggested approaches for facilitating explanatory flows for decision-system design.

Keywords: machine learning, algorithmic decisionmaking, explainability, artificial intelligence

Suggested Citation

Strandburg, Katherine J., Rulemaking and Inscrutable Automated Decision Tools (2019). 119 Columbia Law Review 1851 (2019), NYU School of Law, Public Law Research Paper No. 20-36, Available at SSRN: https://ssrn.com/abstract=3546376

Katherine J. Strandburg (Contact Author)

New York University School of Law ( email )

40 Washington Square South
New York, NY 10012-1099
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
310
Abstract Views
1,276
Rank
178,568
PlumX Metrics