Hacking Generative AI

64 Pages Posted: 3 May 2024

See all articles by Ido Kilovaty

Ido Kilovaty

University of Arkansas - School of Law; Yale University - Law School

Date Written: March 1, 2024


Generative AI platforms, like ChatGPT, hold great promise in enhancing human creativity, productivity, and efficiency. However, generative AI platforms are prone to manipulation. Specifically, they are susceptible to a new type of attack called “prompt injection.” In prompt injection, attackers carefully craft their input prompt to manipulate AI into generating harmful, dangerous, or illegal content as output. Examples of such outputs include instructions on how to build an improvised bomb, how to make meth, how to hotwire a car, and more. Researchers have also been able to make ChatGPT generate malicious code.

This article asks a basic question: do prompt injection attacks violate computer crime law, mainly the Computer Fraud and Abuse Act? This article argues that they do. Prompt injection attacks lead AI to disregard its own hard-coded content generation restrictions, which allows the attacker to access portions of the AI that are beyond what the system’s developers authorized. Therefore, this constitutes the criminal offense of accessing a computer in excess of authorization. Although prompt injection attacks could run afoul of the Computer Fraud and Abuse Act, this article offers ways to distinguish serious acts of AI manipulation from less serious ones, so that prosecution would only focus on a limited set of harmful and dangerous prompt injections.

Keywords: cyber crime, artificial intelligence, cybersecurity, hacking, ai

Suggested Citation

Kilovaty, Ido, Hacking Generative AI (March 1, 2024). Loyola of Los Angeles Law Review, Vol. 58, 2025, Available at SSRN: https://ssrn.com/abstract=4788909

Ido Kilovaty (Contact Author)

University of Arkansas - School of Law ( email )

260 Waterman Hall
Fayetteville, AR 72701
United States

Yale University - Law School ( email )

P.O. Box 208215
New Haven, CT 06520-8215
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Abstract Views
PlumX Metrics