Industrializing Deep Reinforcement Learning for Operational Spare Parts Inventory Management

Abstract

We show how Deep Reinforcement Learning (DRL) can improve industrial-scale operational spare parts inventory management. Spare parts inventory is crucial for the timely maintenance of capital goods. Operational spare parts management requires fast decision-making for complex and large-scale service networks. Therefore, existing work focuses on computationally-light heuristics to maintain tractability, even if that comes at a cost in performance. This is problematic, as lower performance in this context may mean unnecessary downtime on a bottleneck machine, or excessive spending on inventory procurement and expedited shipments. DRL models can theoretically be trained to take near-optimal decisions for such complex problems almost instantaneously, yet training DRL models for industrial-scale inventory systems remains an open challenge. We propose a novel DRL approach that builds on three techniques: global learning to scale over an arbitrary number of SKUs, action space decomposition to cope with a large number of locations, and reward smoothing to efficiently handle stochastic and sparse demand. We demonstrate the effectiveness of our approach on the service network of ASML, a leading company in the semiconductor industry. The results show that our DRL approach outperforms existing methods by 3.8% to 5.1% in terms of cost on a fully connected service network with 10,000 stock keeping units and 60 locations. We show how DRL can improve upon the state-of-the-art in operational spare parts management, and extend the applicability of DRL to industrial-scale inventory systems.

Keywords: Deep Reinforcement Learning, Inventory Management, Service Logistics, Reward Smoothing, Action Space Decomposition, Global Learning