Two-Armed Restless Bandits with Imperfect Information: Stochastic Control and Indexability

50 Pages Posted: 11 May 2013 Last revised: 15 Jul 2023

See all articles by Roland G. Fryer

Roland G. Fryer

Harvard University - Department of Economics; National Bureau of Economic Research (NBER); American Bar Foundation; University of Chicago

Philipp Harms

University of Freiburg - Institut für Mathematische Stochastik

Date Written: May 2013

Abstract

We present a two-armed bandit model of decision making under uncertainty where the expected return to investing in the "risky arm'' increases when choosing that arm and decreases when choosing the "safe'' arm. These dynamics are natural in applications such as human capital development, job search, and occupational choice. Using new insights from stochastic control, along with a monotonicity condition on the payoff dynamics, we show that optimal strategies in our model are stopping rules that can be characterized by an index which formally coincides with Gittins' index. Our result implies the indexability of a new class of "restless'' bandit models.

Suggested Citation

Fryer, Roland G. and Harms, Philipp, Two-Armed Restless Bandits with Imperfect Information: Stochastic Control and Indexability (May 2013). NBER Working Paper No. w19043, Available at SSRN: https://ssrn.com/abstract=2263626

Roland G. Fryer (Contact Author)

Harvard University - Department of Economics ( email )

Littauer Center
Cambridge, MA 02138
United States

National Bureau of Economic Research (NBER)

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

American Bar Foundation

750 N. Lake Shore Drive
Chicago, IL 60611
United States

University of Chicago ( email )

1101 East 58th Street
Chicago, IL 60637
United States

Philipp Harms

University of Freiburg - Institut für Mathematische Stochastik ( email )

Freiburg, 79104
Germany

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
31
Abstract Views
518
PlumX Metrics