Transfer Learning and Textual Analysis of Accounting Disclosures: Applying Big Data Methods to Small(er) Data Sets
Accounting Horizons
49 Pages Posted: 20 Apr 2020 Last revised: 9 Apr 2023
Date Written: March 20, 2020
Abstract
We introduce and apply machine transfer learning methods to analyze accounting disclosures. We use the examples of the new BERT language model and sentiment analysis of quarterly earnings disclosures to demonstrate the key transfer learning concepts of:
(i) pre-training on generic “Big Data”,
(ii) fine-tuning on small accounting data-sets, and
(iii) using a language model that captures context rather than stand-alone words.
Overall, we show that this new approach is easy to implement, uses widely-available and low-cost computing resources, and has superior performance relative to existing textual analysis tools in accounting. We conclude with suggestions for opportunities to apply transfer learning to address important accounting research questions.
Keywords: Disclosure, Earnings Announcement, Machine Learning, Natural Language Processing, Neural Network, Textual Analysis, Transfer Learning
JEL Classification: G31, G32, M21, M41
Suggested Citation: Suggested Citation