GPT has become financially literate: Insights from financial literacy tests of GPT and a preliminary test of how people use it as a source of advice

43 Pages Posted: 11 Mar 2023 Last revised: 26 Mar 2024

See all articles by Paweł Niszczota

Paweł Niszczota

Poznań University of Economics and Business - Humans & AI Laboratory (HAI Lab)

Sami Abbas

affiliation not provided to SSRN

Date Written: August 17, 2023

Abstract

We assess the ability of GPT–a large language model–to serve as a financial robo-advisor for the
masses, by using a financial literacy test. Davinci and ChatGPT based on GPT-3.5 score 66% and
65% on the financial literacy test, respectively, compared to a baseline of 33%. However,
ChatGPT based on GPT-4 achieves a near-perfect 99% score, pointing to financial literacy
becoming an emergent ability of state-of-the-art models. We use the Judge-Advisor System and a
savings dilemma to illustrate how researchers might assess advice-utilization from large language
models. We also present a number of directions for future research.

Keywords: Financial literacy; Robo-advice; Financial advice; Advice-utilization; Large language model; ChatGPT

JEL Classification: D14, G11, G53

Suggested Citation

Niszczota, Paweł and Abbas, Sami, GPT has become financially literate: Insights from financial literacy tests of GPT and a preliminary test of how people use it as a source of advice (August 17, 2023). Available at SSRN: https://ssrn.com/abstract=4384861 or http://dx.doi.org/10.2139/ssrn.4384861

Paweł Niszczota (Contact Author)

Poznań University of Economics and Business - Humans & AI Laboratory (HAI Lab) ( email )

al. Niepodległości 10
Poznań, 61-875
Poland

Sami Abbas

affiliation not provided to SSRN ( email )

No Address Available

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
999
Abstract Views
2,138
Rank
44,637
PlumX Metrics