The AI Economist: Optimal Economic Policy Design via Two-level Deep Reinforcement Learning

37 Pages Posted: 9 Aug 2021 Last revised: 20 Aug 2021

See all articles by Stephan Zheng

Stephan Zheng

Salesforce

Alexander Trott

Salesforce

Sunil Srinivasa

Salesforce

David C. Parkes

Harvard University - Division of Engineering and Applied Sciences

Richard Socher

You.com

Date Written: August 5, 2021

Abstract

AI and reinforcement learning (RL) have improved many areas, but are not yet widely adopted in economic policy design, mechanism design, or economics at large.

At the same time, current economic methodology is limited by a lack of counterfactual data, simplistic behavioral models, and limited opportunities to experiment with policies and evaluate behavioral responses.

Here we show that machine-learning-based economic simulation is a powerful policy and mechanism design framework to overcome these limitations.

The AI Economist is a two-level, deep RL framework that trains both agents and a social planner who co-adapt, providing a tractable solution to the highly unstable and novel two-level RL challenge.

From a simple specification of an economy, we learn rational agent behaviors that adapt to learned planner policies and vice versa.

We demonstrate the efficacy of the AI Economist on the problem of optimal taxation. In simple one-step economies, the AI Economist recovers the optimal tax policy of economic theory. In complex, dynamic economies, the AI Economist substantially improves both utilitarian social welfare and the trade-off between equality and productivity over baselines. It does so despite emergent tax-gaming strategies, while accounting for agent interactions and behavioral change more accurately than economic theory.

These results demonstrate for the first time that two-level, deep RL can be used for understanding and as a complement to theory for economic design, unlocking a new computational learning-based approach to understanding economic policy.

Keywords: Economics, Machine Learning, Reinforcement Learning, Simulation, Income Taxation, Economic Policy

Suggested Citation

Zheng, Stephan and Trott, Alexander and Srinivasa, Sunil and Parkes, David C. and Socher, Richard, The AI Economist: Optimal Economic Policy Design via Two-level Deep Reinforcement Learning (August 5, 2021). Available at SSRN: https://ssrn.com/abstract=3900018 or http://dx.doi.org/10.2139/ssrn.3900018

Stephan Zheng (Contact Author)

Salesforce ( email )

United States

Alexander Trott

Salesforce ( email )

United States

Sunil Srinivasa

Salesforce ( email )

United States

David C. Parkes

Harvard University - Division of Engineering and Applied Sciences ( email )

1875 Cambridge Street
Cambridge, MA 02138
United States

Richard Socher

You.com ( email )

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
40
Abstract Views
144
PlumX Metrics