Sailing through the Dark: Provably Sample-Efficient Inventory Control

Abstract

We consider the important open problems of 1) What is the sample complexity (i.e., how many data samples are needed) of learning nearly optimal policy for multi-stage stochastic inventory control when the underlying demand distribution is initially unknown; and 2) How to compute such a policy when the required number of data samples are given. Starting from the backlog setting, we first propose SAIL, a novel SAmple based Inventory Learning algorithm, and rigorously analyze its sample complexity upper bound. We also complement this by a matching (up to logarithmic factors) lower bound. Then, we extend our results to the more practical lost-sales setting, and provide the first sample complexity result for this more challenging setting with only mild assumptions (that ensures quality data). Our algorithmic results leverage both recent developments of variance reduction techniques for reinforcement learning and the structural properties of the dynamic programming formulation for inventory control settings. Finally we conduct extensive numerical simulations, using both synthetic and real-world datasets, to show that SAIL significantly outperforms competing methods in terms of inventory cost minimization when only historical data samples are available.

Keywords: inventory control, dynamic programming, reinforcement learning, sample complexity

JEL Classification: C63