Experimental Demonstration of Time-Efficient Auto-Calibration of a Vehicle Thermal Management System Using Safe Reinforcement Learning

Abstract

Future automotive powertrains are getting increasingly complex to meet the optimality and robust performance targets in real-world conditions. Traditional map-based control methods require significant calibration effort to achieve optimal performance and real-world robustness under various operating conditions. State-of-the-art model-based control methods reduce the calibration effort by more off-line control development using system models. However, they offer limited real-world robustness and require accurate system models, which are challenging to generate for new and complex powertrain systems. Self Learning Control, i.e., autonomously learning optimal control settings directly on the vehicle, has the potential to overcome the challenges of extensive calibration effort and limited robustness in existing control methods. This paper aims to demonstrate the potential of Reinforcement Learning-based Self Learning control on a vehicle subsystem with safety constraints. To realize Reinforcement Learning on a vehicle, a novel exploration method is applied, which can ensure system safety and minimize experiment time. We demonstrate the exploration method on a vehicle test-bench for the calibration of the reference generator to optimize steady-state operation of the battery electric vehicle thermal system. We showcase the ease of calibration with the proposed method for change in the functional requirement of the system. Moreover, we demonstrate the robustness of the control method under varying ambient conditions. The proposed method achieves the desired heat pump system efficiency, which is within 2% of the true optimum for all the validation operating points. In terms of the calibration effort, there is a significant 69.6% reduction compared to the benchmark map-based control method.

Keywords: Self Learning control, reinforcement learning, Vehicle thermal management, automotive control