Lessons Learned When Teaching Data Analytics with ChatGPT to MBAs in Spring 2023

Abstract

ChatGPT rendered my old data science class obsolete, so I created a new version that leverages large language models. This paper chronicles my experiences incorporating ChatGPT into the MBA classroom. I learned that the chatbot does not blur the lines between A and B students. To maintain a robust grade distribution, I only needed to increase the quizzes' difficulty marginally. I also found that MBA students are well suited to "programming in English"---i.e., expressing the required data transformations in words and then using a large language model to convert their prose into code. I present informal experimental results that suggest that moderately advanced data-wrangling tasks are more manageable for students when they program in English than when they use Excel. The integration of ChatGPT yielded mixed results: both my elective and compulsatory sections seemed more engaged, but only the former gave me higher evaluations---the compulsatory section gave me significantly lower teaching scores. Students from both sections also reported working less outside of class.

Keywords: ChatGPT, large language models, data science education