The Rise of Generative AI for Tax Research

Tax Notes Federal, May 29, 2023, p. 1509

15 Pages Posted: 21 Jun 2023

See all articles by Benjamin Alarie

Benjamin Alarie

University of Toronto - Faculty of Law; Vector Institute for Artificial Intelligence

Kim Condon

Blue J Legal

Susan Massey

Blue J Legal

Christopher Yan

Blue J Legal

Date Written: May 29, 2023

Abstract

ChatGPT has exploded into the popular consciousness. Generative artificial intelligence, in the form of natural language chatbots, is newly poised to significantly shift how tax professionals conduct research. These emerging advanced AI systems can analyze vast amounts of text and generate coherent, contextually relevant responses.

As various industries continue to adopt AI-powered tools, lawyers need to understand and adapt to these new technologies. As Jason Chen has written in Tax Notes, “It is only by incorporating these types of technological advances into daily business practices that modern tax professionals can gain valuable advantages over others in the increasingly competitive world of taxation.”

In this article, we will consider the benefits of using generative AI, examine the concerns, and discuss ways that it can be improved. To illustrate these considerations, we submitted a series of tax law questions to three different generative AI models: ChatGPT (based on GPT-3.5), ChatGPT Plus (based on GPT-4), and Ask Blue J. The questions were selected to represent a variety of tax law areas and to present the types of research questions that a typical tax law practitioner might encounter in their day-to-day practice.

For this experiment we posed identical questions to ChatGPT and Ask Blue J and recorded their responses. We tested the questions against the GPT-3.5 and GPT-4 versions of ChatGPT. We primed each conversation with ChatGPT using the following instruction: “I want you to answer all questions as they apply in the context of U.S. federal income tax law. Please provide IRS resources along with your answers.” This additional instruction was provided to ChatGPT only because Ask Blue J is already fine-tuned to focus on tax and to provide sources with its responses. Providing the priming instruction to the GPT models ensures a fair comparison. Other than this introductory prompt, our questions to each chatbot were the same. The results highlight the importance of developers and users engaging with this technology thoughtfully when harnessing it for tax research purposes.

Keywords: ChatGPT, generative AI, artificial intelligence, tax research

JEL Classification: H0

Suggested Citation

Alarie, Benjamin and Condon, Kim and Massey, Susan and Yan, Christopher, The Rise of Generative AI for Tax Research (May 29, 2023). Tax Notes Federal, May 29, 2023, p. 1509, Available at SSRN: https://ssrn.com/abstract=4476510

Benjamin Alarie (Contact Author)

University of Toronto - Faculty of Law ( email )

Jackman Law Building
78 Queen's Park
Toronto, Ontario M5S 2C5
Canada
416-946-8205 (Phone)
416-978-7899 (Fax)

HOME PAGE: http://www.benjaminalarie.com

Vector Institute for Artificial Intelligence ( email )

Kim Condon

Blue J Legal ( email )

Susan Massey

Blue J Legal

Christopher Yan

Blue J Legal ( email )

325 FRONT ST W
TORONTO, ON M8Z2C3
Canada

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
792
Abstract Views
2,081
Rank
63,464
PlumX Metrics