Normal Autonomous Accidents: What Happens When Killer Robots Fail?

Abstract

Over the past decade there has been much written on lethal autonomous weapons systems (LAWS) commonly known as “killer robots”. This includes legal, ethical and moral concerns as well as issues related to responsibility. And yet, for all of the discussion directed to concerns about what happens when something goes wrong, there is less attention paid as to how it will go wrong. The main difference between LAWS and our current weapons systems is the freedom they will have to make decisions on the battlefield, creating unique challenges for how we regulate such weapons. Not only is it potentially more difficult to verify the reliable behavior of LAWS, but the consequences of an accident could potentially be more severe.

As such, starting from the assumption that LAWS will be significantly more complex than our present weapons systems, this paper uses Charles Perrow’s Normal Accident Theory (and its critics) to explore the concerns raised over LAWS through the lens of system failure. Focusing on failure provides new insights into problems related to risk mitigation strategies (such as weapons reviews) and responsibilities (chain of command). Moreover, it also points to preliminary steps the international community can take in regulating future weapons systems

Keywords: Normal Accident Theory, Artificial Intelligence, Autonomous Weapons, International Law, Laws of War, International Humanitarian Law, 1980 Conventional Weapons Convention, Military Technology