Can Deep Reinforcement Learning Improve Inventory Management? Performance on Dual Sourcing, Lost Sales and Multi-Echelon Problems

Abstract

(Forthcoming in Manufacturing & Service Operations Management)

Problem definition: Is Deep Reinforcement Learning (DRL) effective at solving inventory problems?

Academic/practical relevance: Given that DRL has successfully been applied in computer games and robotics, supply chain researchers and companies are interested in its potential in inventory management. We provide a rigorous performance evaluation of DRL in three classic and intractable inventory problems: lost sales, dual-sourcing and multi-echelon inventory management.

Methodology: We model each inventory problem as a Markov Decision Process and apply and tune the Asynchronous Advantage Actor Critic (A3C) DRL algorithm for a variety of parameter settings.

Results: We demonstrate that the A3C algorithm can match performance of state-of-the-art heuristics and other approximate dynamic programming methods. While the initial tuning was computationally- and time-demanding, only small changes to the tuning parameters were needed for the other studied problems.

Managerial implications: Our study provides evidence that DRL can effectively solve inventory problems. This is especially promising when problem-dependent heuristics are lacking. Yet generating structural policy insight or designing specialized policies that are (ideally provably) near optimal remains desirable.

Keywords: artificial intelligence, deep reinforcement learning, inventory control, dual sourcing, lost sales, multi-echelon

JEL Classification: M11