Marrying Stochastic Gradient Descent with Bandits: Learning Algorithms for Inventory Systems with Fixed Costs

44 Pages Posted: 7 Mar 2019

See all articles by Hao Yuan

Hao Yuan

University of Michigan at Ann Arbor - Department of Industrial and Operations Engineering

Qi Luo

University of Michigan at Ann Arbor - Department of Industrial and Operations Engineering

Cong Shi

University of Michigan at Ann Arbor - Department of Industrial and Operations Engineering

Date Written: February 1, 2019

Abstract

We consider a periodic-review single-product inventory system with fixed cost under censored demand. Under full demand distributional information, it is well-known that the celebrated $(s,S)$ policy is optimal. In this paper, we assume the firm does not know the demand distribution a priori, and makes adaptive inventory ordering decision in each period based only on the past sales (a.k.a. censored demand) data. The standard performance measure is regret, which is the cost difference between a feasible learning algorithm and the clairvoyant (full-information) benchmark. Compared with prior literature, the key difficulty of this problem lies in the loss of joint convexity of the objective function, due to the presence of fixed cost. We develop a nonparametric learning algorithm termed the $(\delta, S)$ policy that combines the powers of stochastic gradient descent, bandit controls, and simulation-based methods in a seamless and non-trivial fashion. We prove that the cumulative regret is $O(\log T\sqrt{T})$, which is provably tight up to a logarithmic factor. We also develop several technical results that are of independent interest. We believe that the framework developed could be widely applied to learning other important stochastic systems with partial convexity in the objectives.

Keywords: inventory, fixed costs, censored demand, nonparametric, learning algorithms, regret analysis

Suggested Citation

Yuan, Hao and Luo, Qi and Shi, Cong, Marrying Stochastic Gradient Descent with Bandits: Learning Algorithms for Inventory Systems with Fixed Costs (February 1, 2019). Available at SSRN: https://ssrn.com/abstract=3329611

Hao Yuan

University of Michigan at Ann Arbor - Department of Industrial and Operations Engineering ( email )

1205 Beal Avenue
Ann Arbor, MI 48109
United States

Qi Luo

University of Michigan at Ann Arbor - Department of Industrial and Operations Engineering ( email )

1205 Beal Avenue
Ann Arbor, MI 48109
United States

Cong Shi (Contact Author)

University of Michigan at Ann Arbor - Department of Industrial and Operations Engineering ( email )

1205 Beal Avenue
Ann Arbor, MI 48109
United States

Register to save articles to
your library

Register

Paper statistics

Downloads
118
Abstract Views
773
rank
238,072
PlumX Metrics