GPT has become financially literate: Insights from financial literacy tests of GPT and a preliminary test of how people use it as a source of advice
43 Pages Posted: 11 Mar 2023 Last revised: 26 Mar 2024
Date Written: August 17, 2023
Abstract
We assess the ability of GPT–a large language model–to serve as a financial robo-advisor for the
masses, by using a financial literacy test. Davinci and ChatGPT based on GPT-3.5 score 66% and
65% on the financial literacy test, respectively, compared to a baseline of 33%. However,
ChatGPT based on GPT-4 achieves a near-perfect 99% score, pointing to financial literacy
becoming an emergent ability of state-of-the-art models. We use the Judge-Advisor System and a
savings dilemma to illustrate how researchers might assess advice-utilization from large language
models. We also present a number of directions for future research.
Keywords: Financial literacy; Robo-advice; Financial advice; Advice-utilization; Large language model; ChatGPT
JEL Classification: D14, G11, G53
Suggested Citation: Suggested Citation