Smoothness-Adaptive Contextual Bandits

Operations Research (forthcoming)

78 Pages Posted: 14 Sep 2021 Last revised: 14 Oct 2021

See all articles by Yonatan Gur

Yonatan Gur

Stanford Graduate School of Business

Ahmadreza Momeni

Stanford University

Stefan Wager

Stanford University - Department of Statistics

Date Written: July 25, 2021

Abstract

We study a non-parametric multi-armed bandit problem with stochastic covariates, where a key complexity driver is the smoothness of payoff functions with respect to covariates. Previous studies have focused on deriving minimax-optimal algorithms in cases where it is a priori known how smooth the payoff functions are. In practice, however, the smoothness of payoff functions is typically not known in advance, and misspecification of smoothness may severely deteriorate the performance of existing methods. In this work, we consider a framework where the smoothness of payoff functions is not known, and study when and how algorithms may adapt to unknown smoothness. First, we establish that designing algorithms that adapt to unknown smoothness of payoff functions is, in general, impossible. However, under a self-similarity condition (which does not reduce the minimax complexity of the dynamic optimization problem at hand), we establish that adapting to unknown smoothness is possible, and further devise a general policy for achieving smoothness-adaptive performance. Our policy infers the smoothness of payoffs throughout the decision-making process, while leveraging the structure of off-the-shelf non-adaptive policies. We establish that for problem settings with either differentiable or non-differentiable payoff functions, this policy matches (up to a logarithmic scale) the regret rate that is achievable when the smoothness of payoffs is known a priori.

Keywords: Contextual multi-armed bandits, Holder smoothness, self-similarity, non-parametric confidence intervals, non-parametric estimation, experiment design

JEL Classification: C44, C45, C9

Suggested Citation

Gur, Yonatan and Momeni, Ahmadreza and Wager, Stefan, Smoothness-Adaptive Contextual Bandits (July 25, 2021). Operations Research (forthcoming), Available at SSRN: https://ssrn.com/abstract=3893198 or http://dx.doi.org/10.2139/ssrn.3893198

Yonatan Gur (Contact Author)

Stanford Graduate School of Business ( email )

655 Knight Way
Stanford, CA 94305-5015
United States

Ahmadreza Momeni

Stanford University ( email )

Stanford, CA 94305
United States

Stefan Wager

Stanford University - Department of Statistics ( email )

Stanford, CA 94305
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
20
Abstract Views
114
PlumX Metrics