‘Words Are Flowing Out Like Endless Rain Into a Paper Cup’: ChatGPT & Law School Assessments

40 Pages Posted: 15 Feb 2023

See all articles by Stuart Hargreaves

Stuart Hargreaves

The Chinese University of Hong Kong (CUHK) - Faculty of Law

Date Written: January 2023

Abstract

ChatGPT is a sophisticated large-language model able to answer high-level questions in a way that is undetectable by conventional plagiarism detectors. Concerns have been raised it poses a significant risk of academic dishonesty in ‘take-home’ assessments in higher education. To evaluate this risk in the context of legal education, this project had ChatGPT generate answers to twenty-four different exams from an English-language law school based in a common law jurisdiction. It found that the system performed best on exams that were essay-based and asked students to discuss international legal instruments or general legal principles not necessarily specific to any jurisdiction. It performed worst on exams that featured problem-style or “issue spotting” questions asking students to apply an invented factual scenario to local legislation or jurisprudence. While the project suggests that for the most part conventional law school assessments are for the time being relatively immune from the threat ChatGPT brings, this is unlikely to remain the case as the technology advances. However, rather than attempt to block students from using AI as part of learning and assessment, this paper instead proposes three ways students may be taught to use it in appropriate and ethical ways. While it is clear that ChatGPT and similar AI technologies will change how universities teach and assess (across disciplines), a solution of prevention or denial is no solution at all.

Keywords: law, ChatGPT, AI, artificial intelligence, higher education, law schools, assessments, plagiarism

JEL Classification: K10, I23,

Suggested Citation

Hargreaves, Stuart, ‘Words Are Flowing Out Like Endless Rain Into a Paper Cup’: ChatGPT & Law School Assessments (January 2023). The Chinese University of Hong Kong Faculty of Law Research Paper No. 2023-03, Available at SSRN: https://ssrn.com/abstract=4359407 or http://dx.doi.org/10.2139/ssrn.4359407

Stuart Hargreaves (Contact Author)

The Chinese University of Hong Kong (CUHK) - Faculty of Law ( email )

6F Lee Shau Kee Building
Shatin, New Territories
Shatin, New Territories
Hong Kong

HOME PAGE: http://www.law.cuhk.edu.hk/en/people/info.php?id=20

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
1,270
Abstract Views
4,214
Rank
31,165
PlumX Metrics