Bimodal Temporal Modeling Reinforcement Learning with Safety Mechanism for Highway Lane Change in Mixed Traffic

Abstract

The differences in driving behavior between Connected and Autonomous Vehicles (CAVs) and Human-Drive Vehicles (HDVs) have a significant impact on the safety and efficiency of traffic in complex mixed traffic environments. Ensuring that CAVs can safely and efficiently change lanes remains a challenging issue. Previous research has focused on designing reward functions or enforcing basic constraints to incorporate safety and efficiency. However, these methods often do not model the relationships between vehicles in the observation space, and thus cannot fully understand the interactions between vehicles and capture potential hazards, lacking response to and potential interaction with other vehicles on the target lane. Additionally, biased sample selection may lead to a lack of understanding of time or historical trajectories, making the decision of the model unstable. To address these issues, we propose a bi-modal temporal modeling reinforcement learning algorithm, combined with a lane-changing safety mechanism, allowing CAVs to model bi-modal information of observed states and vehicle relationships. The temporal module extracts representation information from historical experiences while assessing the collision risk with other vehicles during the lane-changing process, evaluating the safety of the lane-changing behavior and guiding the policy network to select the optimal action. We train and evaluate our approach based on training rewards and average vehicle speed in three different traffic scenarios, testing the collision rate and average travel distance of the method. The test results demonstrate that our approach significantly reduces the collision rate and improves the average vehicle speed in all three traffic scenarios, proving the effectiveness of our method.

Keywords: highway lane change, bimodal temporal modeling, Reinforcement Learning, safety mechanism, mixed traffic