Developing energy management strategies (EMSs) for different types of hybrid electric vehicles (HEVs) is a time-consuming and laborious task for automotive engineers. Experienced engineers can reduce the developing cycle by exploiting the commonalities between different types of HEV EMSs. Aiming at improving the efficiency of HEV EMSs development automatically, this paper proposes a transfer learning based method to achieve the cross-type knowledge transfer between deep reinforcement learning (DRL) based EMSs. Specifically, knowledge transfer among four significantly different types of HEVs is studied. We first use massive driving cycles to train a DRL-based EMS for Prius. Then the parameters of its deep neural networks, wherein the common knowledge of energy management is captured, are transferred into EMSs of a powersplit bus, a series vehicle and a series-parallel bus. Finally, the parameters of 3 different HEV EMSs are fine-tuned in a small dataset. Simulation results indicate that, by incorporating transfer learning (TL) into DRL-based EMS for HEVs, an average 70% gap from the baseline in respect of convergence efficiency has been achieved. Our study also shows that TL can transfer knowledge between two HEVs that have significantly different structures. Overall, TL is conducive to boost the development process for HEV EMS.

Modern automotive systems have been equipped with a highly increasing number of onboard computer vision hardware and software, which are considered to be beneficial for achieving eco-driving. This article combines computer vision and deep reinforcement learning (DRL) to improve the fuel economy of hybrid electric vehicles. The proposed method is capable of autonomously learning the optimal control policy from visual inputs. The state-of-the-art convolutional neural networks-based object detection method is utilized to extract available visual information from onboard cameras. The detected visual information is used as a state input for a continuous DRL model to output energy management strategies. To evaluate the proposed method, we construct 100 km real city and highway driving cycles, in which visual information is incorporated. The results show that the DRL-based system with visual information consumes 4.3-8.8% less fuel compared with the one without visual information, and the proposed method achieves 96.5% fuel economy of the global optimum-dynamic programming.

Reinforcement learning (RL) based energy management is one of the current hot-spots of hybrid electric vehicles. Recent advances in RL based energy management focus on energy-saving performance but less considers the constrained setting for training safety. This paper proposes an RL framework named Coach-Actor-Double Critic (CADC) for the optimization of energy management considered as the constrained Markov Decision Process (CMDP). A bi-level onboard controller includes a neural network (NN) based strategy Actor and rule-based strategy Coach for online energy management. Once the output of Actor exceeds the constrained range of feasible solutions, Coach would take charge of energy management to ensure safety. By using Lagrangian relaxation, the optimization for constrained Markov decision process transforms into an unconstrained dual problem to minimize the energy consumption while minimizing the Coach participation. The parameters of Actor are updated in a manner of policy gradient through RL training with the Lagrangian value function. Double Critic with the same structure synchronously estimate the value function to avoid overestimate bias. Several experiments with the bus trajectories data demonstrate the optimality, self-learning ability, and adaptability of CADC. The results indicate that CADC outperforms the existing RL based strategies and reaches above 95% energy-saving rate of the offline global optimum.

Car-following models have been widely applied and made remarkable achievements in traffic engineering. However, the traffic micro-simulation accuracy of car-following models in a platoon level, especially during traffic oscillations, still needs to be enhanced. Rather than using traditional individual car-following models, we proposed a new trajectory generation approach to generate platoon level trajectories given the first leading vehicle's trajectory. In this article, we discussed the temporal and spatial error propagation issue for the traditional approach by a car following block diagram representation. Based on the analysis, we pointed out that error comes from the training method and the model structure. In order to fix that, we adopt two improvements on the basis of the traditional LSTM-based car-following model. We utilized a scheduled sampling technique during the training process to solve the error propagation in the temporal dimension. Furthermore, we developed a unidirectional interconnected LSTM model structure to extract trajectories features from the perspective of the platoon. As indicated by the systematic empirical experiments, the proposed novel structure could efficiently reduce the temporal-spatial error propagation. Compared with the traditional LSTM-based car-following model, the proposed model has almost 40% less error. The findings will benefit the design and analysis of micro-simulation for platoon-level car-following models.