Artificial Intelligence’s Capabilities, Limitations, and Impact on Accounting Education: Investigating ChatGPT’s Performance on Educational Accounting Cases

33 Pages Posted: 10 May 2023

See all articles by Xu (Joyce) Cheng

Xu (Joyce) Cheng

Auburn University

Ryan Dunn

Auburn University - School of Accountancy

Travis Holt

Auburn University

Kerry Inger

Auburn University - School of Accountancy

J. Gregory Jenkins

Auburn University - School of Accountancy

Jefferson Jones

Auburn University

James H. Long

Auburn University - School of Accountancy

Tina M. Loraas

Auburn University

Mollie Mathis

Auburn University

Jonathan D. Stanley

Auburn University - School of Accountancy

David A. Wood

Brigham Young University - School of Accountancy

Date Written: April 27, 2023

Abstract

This study evaluates the capabilities of ChatGPT models 3.5 and 4 to provide solutions to eight educational accounting cases. We find that ChatGPT’s ability to provide accurate solutions varies depending on the type of case requirement and topical area of accounting, with better performance on tasks requiring elements that require explanation, application of rules, and ethical evaluation. However, ChatGPT performs poorly on tasks that require financial statement creation, journal entries, or software use. Our study also finds that detection tools provided by ChatGPT's developer are ineffective in identifying text created by Artificial Intelligence Text Generators (AITG). We suggest ways in which instructors can develop and structure assignments to reduce the effectiveness of AITGs in subverting the learning process, and discuss ways in which AITGs can be incorporated into assignment requirements to help students attain, desired educational outcomes.

Keywords: Artificial intelligence, text generators, Chatbots, ChatGPT, accounting education

JEL Classification: M40, M41, M42, M49

Suggested Citation

Cheng, Xu (Joyce) and Dunn, Ryan and Holt, Travis and Inger, Kerry and Jenkins, J. Gregory and Jones, Jefferson and Long, James H. and Loraas, Tina M. and Mathis, Mollie and Stanley, Jonathan D. and Wood, David A., Artificial Intelligence’s Capabilities, Limitations, and Impact on Accounting Education: Investigating ChatGPT’s Performance on Educational Accounting Cases (April 27, 2023). Available at SSRN: https://ssrn.com/abstract=4431202 or http://dx.doi.org/10.2139/ssrn.4431202

Xu (Joyce) Cheng

Auburn University ( email )

Ryan Dunn

Auburn University - School of Accountancy ( email )

415 West Magnolia Avenue
301 Lowder Business Building
Auburn, AL 36849
United States

Travis Holt

Auburn University ( email )

415 West Magnolia Avenue
Auburn, AL 36849
United States

Kerry Inger

Auburn University - School of Accountancy ( email )

415 West Magnolia Avenue
301 Lowder Business Building
Auburn, AL 36849
United States

J. Gregory Jenkins

Auburn University - School of Accountancy ( email )

415 West Magnolia Avenue
301 Lowder Business Building
Auburn, AL 36849
United States

Jefferson Jones

Auburn University

James H. Long

Auburn University - School of Accountancy ( email )

301 Lowder Hall
Auburn University, AL 36849
United States
(334) 844-6214 (Phone)

HOME PAGE: http://www.jameshlong.com

Tina M. Loraas

Auburn University ( email )

415 West Magnolia Avenue
Auburn, AL 36849
United States

Mollie Mathis

Auburn University ( email )

309A, Comer Hall
Auburn, AL 36849
United States

Jonathan D. Stanley

Auburn University - School of Accountancy ( email )

415 West Magnolia Avenue
Auburn, AL 36849
United States

David A. Wood (Contact Author)

Brigham Young University - School of Accountancy ( email )

518 TNRB
Brigham Young University
Provo, UT 84602
United States
801-422-8642 (Phone)
801-422-0621 (Fax)

HOME PAGE: http://marriottschool.byu.edu/employee/employee.cfm?emp=daw44

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
522
Abstract Views
1,183
Rank
103,074
PlumX Metrics