Reinforcement learning (RL) methods are commonly regarded as effective solutions for designing intelligent driving policies. Nonetheless, even if the RL policy is converged after training, it is notoriously difficult to ensure safety. In particular, RL policy is susceptible to insecurity in the presence of long-tail or unseen traffic scenarios, i.e. , out-of-distribution test data. Therefore, the design of the RL-based decision-making method must account for this shift in distribution. This paper proposes a robust decision-making framework for autonomous driving on the highway to improve driving safety. First, a Deep Deterministic Policy Gradient (DDPG)-based RL policy that directly maps observations to actions is constructed. Subsequently, the model uncertainty of the DDPG policy is evaluated at runtime to quantify the policy's reliability and identify unseen scenarios. In addition, a complementary principle-based policy is developed using the intelligent driver model (IDM) and the model for minimizing overall braking induced by lane changes (MOBIL). It will take over the DDPG policy when encountering unseen scenarios to guarantee a lower-bound performance of the decision-making system. Finally, the proposed method is implemented on an embedded system, i.e. , NVIDIA Jetson AGX Xavier, and out-of-training distribution challenging cases are considered in the experiment, i.e. , observation with sensor noise, traffic density increasing significantly, objects falling from the front vehicle, and road construction causing temporal changes in road structure. Results indicate that the proposed framework outperforms state-of-the-art benchmarks. Additionally, the code is provided https://github.com/Kayne0401/Robust-Decision-Making-Framework .
12 July 2023 in IEEE Transactions on Vehicular Technology
Safety is the foremost concern for autonomous platooning. The vehicle-to-vehicle (V2V) communication delays and the sudden appearance of obstacles will trigger the safety of the intended functionality (SOTIF) issues for autonomous platooning. This research proposes a holistic robust motion controller framework (MCF) for an intelligent and connected vehicle platoon system. The MCF utilizes a hierarchical structure to resolve the longitudinal string stability and the lateral control problem under the complex driving environment and time-varying communication delays. Firstly, the H-infinity feedback controller is developed to ensure the robustness of the platoon under time-varying communication delay in the upper-level coordination layer (UCL). The output from UCL will be delivered to the lower-level motion-planning layer (LML) as reference signals. Secondly, the model predictive control (MPC) algorithm is implemented in the LML to achieve multi-objective control, which comprehensively considers the reference signals, the artificial potential field, and multiple vehicle dynamics constraints. Furthermore, three critical scenarios are co-simulated for case studies, including platooning under time-varying communication delay, merging, and obstacle avoidance scenarios. The simulation results indicate that, compared with single-structure MPC, the proposed MCF can offer a better suppression on position error propagation, and get improvements on maximum position error in the three scenarios by 19.2%, 59.8%, and 15.3%, respectively. Lastly, the practicability and effectiveness of the proposed MCF are verified via the hardware-in-the-loop experiment. The average conducting time of the proposed method on the Speedgoat real-time target machine is 1.1 milliseconds, which meets the real-time requirements.
22 September 2023 in Accident Analysis & Prevention
Interacting with diverse and stochastic traffic participants is a critical challenge for autonomous vehicles (AVs), as it necessitates advanced decision-making systems to replicate the natural adaptability of human drivers. In particular, navigating safely and efficiently in dense traffic scenarios poses a significant challenge for decision-making, which is inherently an interactive task, i.e., nearby traffic participants will influence AVs’ action, and vice versa. Decision-making solutions that rely solely on unidirectional interaction schemes, neglecting the mutual influence between AVs and other traffic participants, may lead to overly defensive behaviors or the “freezing robot problem”. In recent years, researchers have been increasingly focused on incorporating bidirectional interactions into the decision-making process to make safe, intelligent, and socially compatible decisions. Currently, a comprehensive review of interaction-aware decision-making techniques remains lacking. To this end, this paper aims to provide a systematic review of interaction-aware decision-making methodologies for autonomous driving. Specifically, this paper analyzes the challenges in considering bidirectional interactions between AVs and other traffic participants. In addition, the state-of-the-art techniques for interaction-aware decision-making solutions are reviewed. More importantly, simulation and benchmarks for interaction-aware decision-making validation are also presented. Finally, research perspectives are highlighted to facilitate future studies for interaction-aware decision-making policy design.