Optimizing Timber Supply Chain Operations with Deep Reinforcement Learning-Based Inventory Management

Abstract

Timber supply chain, an essential part of the forestry sector, connects the landowners and the mills to supply diverse wood products to various industries. This supply chain deals with several challenges, including fluctuating market demands, seasonal variations, and natural disruptions such as hurricanes, which can significantly disrupt operations and elevate costs (e.g., inventory-related expenses). In response to these stochastic and dynamic barriers, the current work presents a smart agent-based modeling approach applying Deep Reinforcement Learning (DRL). To be more specific, a generic mathematical model is first formulated to capture the interactions between one mill and multiple landowners, considering a seasonally-adjusted stochastic demand and dynamic hurricane-induced timber prices. The model is later structured in a Markov Decision Process (MDP) framework to allow the mill as the agent to learn inventory policies through iterative optimization. Finally, Proximal Policy Optimization (PPO), a state-of-the-art policy gradient algorithm, is implemented to solve the proposed model by extracting optimal inventory control policies in a stochastic environment. We evaluated the developed method by considering a general case study based on the timberlands in the southern U.S. The initial results confirm the proposed method’s ability to train the mill to optimize its ordering decisions to maintain sufficient stock to meet market needs while minimizing excess inventory and other inventory-related costs. The model is also tested through various experiments, such as variations in the annual timber required by the mill, uncertainty in demand, and varying costs, such as holding costs.

Keywords: Timber supply chain, Inventory Management, agent-based modeling, hurricane risk management, deep learning, reinforcement learning