In order to improve the efficiency and comfort of autonomous vehicles while ensuring safety, the decision algorithm needs to interact with human drivers, infer the most probable behavior and then makes advantageous decision. This paper proposes a Nash-Q learning based motion decision algorithm to consider the interaction. First, the local trajectory of surrounding vehicle is predicted by kinematic constraints, which can reflect the short-term motion trend. Then, the future action space is built based the predicted local trajectory that consists of five basis actions. With that, the Nash-Q learning process can be implemented by the game between these basis actions. By elimination of strictly dominated actions and the Lemke-Howson method, the autonomous vehicle can decide the optimal action and infer the behavior of surrounding vehicle. Finally, the lane merging scenario is built to test the performance contrast to the existing methods. The driver in loop experiment is further designed to verify the interaction performance in multi-vehicle traffic. The results show that the Nash-Q learning based algorithm can improve the efficiency and comfort by 15.75% and 20.71% to the Stackelberg game and the no-interaction method respectively while the safety is ensured. It can also make real-time interaction with human drivers in multi-vehicle traffic.
Driver perception, decision, and control behaviors are easily affected by traffic conditions and driving style, showing the tendency of randomness and personalization. Brake intention and intensity are integrated and control-oriented parameters that are crucial to the development of an intelligent braking system. In this paper, a composite machine learning approach was proposed to predict driver brake intention and intensity with a proper prediction horizon. Various driving data were collected from Controller Area Network (CAN) bus under a real driving condition, which mainly contained urban and rural road types. ReliefF and RReliefF (they don’t have abbreviations) algorithms were employed as feature subset selection methods and applied in a prepossessing step before the training. The rank importance of selected predictors exhibited different trends or even negative trends when predicting brake intention and intensity. A soft clustering algorithm, Fuzzy C-means, was adopted to label the brake intention into categories, namely slight, medium, intensive, and emergency braking. Data sets with misplaced labels were used for training of an ensemble machine learning method, random forest. It was validated that brake intention could be accurately predicted 0.5 s ahead. An open-loop nonlinear autoregressive with external input (NARX) network was capable of learning the long-term dependencies in comparison to the static neural network and was suggested for online recognition and prediction of brake intensity 1 s in advance. As system redundancy and fault tolerance, a close-loop NARX network could be adopted for brake intensity prediction in the case of possible sensor failure and loss of CAN message.
This study presents a multicriteria energy management strategy (EMS) for hybrid energy sources (HES) composed of fuel cell/battery/supercapacitor hybrid power system for logistics trucks, which uses a model-free deep reinforcement learning (DRL) algorithm, namely deterministic strategy gradient (DDPG), to improve the portability and reusability of the system. The proposed EMS is capable of reducing the hydrogen consumption cost, the degradation of fuel cell and battery, as well as sustaining the state of charge (SOC) of battery and supercapacitor. The results of the study found that the total cost was reduced by 9.5% compared to equivalence consumption minimization strategy (ECMS) based EMS under the WLTP driving cycle. A novel deep transfer learning (DTL) based framework for DRL-based EMS is further elaborated and evaluated by four metrics. Two DTL techniques including policy transfer and experience transfer are leveraged to transfer the EMS from original logistic truck to a B-class passenger car powered by fuel cell and battery. The results indicate that the proposed DTL framework is an appropriate approach to transfer EMSs from different vehicle model with various power topology. The convergence speed of DTL-based EMS is apparently accelerated over 50% in comparison to DRL-based EMS. Besides, the fuel optimality, robustness, convergence, and generalization stability for DTL-based EMS is also improved.
29 September 2024 in Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering
The Proton Exchange Membrane Fuel Cell (PEMFC) system, as an efficient, zero-pollution emission, and high-efficiency energy conversion device, is particularly suitable for heavy-duty transportation. Considering the limited power of the current PEMFC system, this study focuses on a hybrid fuel cell heavy-duty truck with dual stacks. Considering the optimal operating range of the PEMFC system and the health status of the lithium-ion battery pack, an energy management strategy (EMS) based on the Deep Deterministic Policy Gradient (DDPG) algorithm is established. This strategy optimizes the hydrogen consumption cost, degradation costs of lithium-ion battery and fuel cells. Additionally, a dynamic programming (DP) and an equivalent consumption minimization strategy (ECMS) based EMSs are established. The power allocation and overall transportation costs under the CSC driving cycle are compared and verified for the three EMSs. The results show that the DDPG algorithm provides a more optimal power allocation for the dual-stack fuel cells, improving the overall system efficiency, enhancing the lifecycle of heavy-duty trucks, and significantly reducing overall travel costs.