The Usefulness of ChatGPT for Textual Analysis of Annual Reports

Abstract

Can predictive AI models be successfully deployed to help investors process complex financial information? We answer this question by examining the usefulness of ChatGPT generated sentiment and complexity scores for a sample of UK annual reports. We document that both measures contain economically significant value-relevant information as captured by their association with (i) price reactions to annual report announcements and (ii) future levels and changes in profitability. Further, both measures predict dispersion in investor beliefs suggesting they capture differences in how investors process textual content of annual reports. The results suggest that investors can employ predictive AI models, such as ChatGPT, to aid them in analyzing textual characteristics of annual reports.

Keywords: AI; ChatGPT; textual analysis; sentiment analysis; annual reports

JEL Classification: C45, D80, G3, G11, G12, G14, M41