Can Deep Reinforcement Learning Improve Inventory Management? Performance on Dual Sourcing, Lost Sales and Multi-Echelon Problems
Manufacturing & Service Operations Management
35 Pages Posted: 3 Jan 2019 Last revised: 16 Nov 2021
Date Written: July 2, 2021
(Forthcoming in Manufacturing & Service Operations Management)
Problem definition: Is Deep Reinforcement Learning (DRL) effective at solving inventory problems?
Academic/practical relevance: Given that DRL has successfully been applied in computer games and robotics, supply chain researchers and companies are interested in its potential in inventory management. We provide a rigorous performance evaluation of DRL in three classic and intractable inventory problems: lost sales, dual-sourcing and multi-echelon inventory management.
Methodology: We model each inventory problem as a Markov Decision Process and apply and tune the Asynchronous Advantage Actor Critic (A3C) DRL algorithm for a variety of parameter settings.
Results: We demonstrate that the A3C algorithm can match performance of state-of-the-art heuristics and other approximate dynamic programming methods. While the initial tuning was computationally- and time-demanding, only small changes to the tuning parameters were needed for the other studied problems.
Managerial implications: Our study provides evidence that DRL can effectively solve inventory problems. This is especially promising when problem-dependent heuristics are lacking. Yet generating structural policy insight or designing specialized policies that are (ideally provably) near optimal remains desirable.
Keywords: artificial intelligence, deep reinforcement learning, inventory control, dual sourcing, lost sales, multi-echelon
JEL Classification: M11
Suggested Citation: Suggested Citation