A Rollout Heuristic-Reinforcement Learning Hybrid Algorithm for Disassembly Sequence Planning with Uncertain Depreciation Condition and Diversified Recovering Strategies

Abstract

Disassembly is one of the crucial aspects of green manufacturing. For the end-of-life products, an effective disassembly sequence planning method can enhance recovery value and mitigate the negative consequences of resource depletion and waste generation. However, both the uncertainty of product depreciation condition and the NP-hard characteristics (including the determination of disassembly sequences and the selection of recovering strategies of subassemblies) of the disassembly sequence planning results in difficulties to determine the optimal/near-optimal disassembly solutions. To address these challenges, this work establishes an extended petri net that considers diversified recovering strategies of each subassembly caused by uncertain product depreciation condition. Then, a rollout heuristic-reinforcement learning hybrid algorithm that integrates a rollout decision rule into the reinforcement learning procedure is proposed to rapidly find the high-quality disassembly solutions based on the extended petri net, in which the uncertainty of disassembly information is tackled by training disassembly samples and the global exploration capability of the learning procedure is significantly improved by using the rollout decision rule. Finally, a hybrid Li-ion battery pack of Audi A3 Sportback e-tron is selected as the case study to verify the performance of the proposed algorithm, and the experimental results indicate that our rollout decision rule-based reinforcement learning algorithm are significantly easier to find the best disassembly solutions comparing with other existing methods.

Keywords: disassembly sequence planning, uncertain depreciation condition, rollout heuristic, reinforcement learning, diversified recovering strategies