It is essential to develop proper energy management strategies (EMSs) with broad adaptability for hybrid electric vehicles (HEVs). This paper utilizes deep reinforcement learning (DRL) to develop EMSs for a series HEV (SHEV) due to DRL's advantages of requiring no future driving information in derivation and good generalization in solving energy management problem formulated as a Markov decision process (MDP). History cumulative trip information is also integrated for effective State of Charge (SoC) guidance in DRL-based EMSs. The proposed method is systematically introduced from offline training to online applications; its learning ability, optimality, and generalization are validated by comparisons with fuel economy benchmark optimized by dynamic programming (DP), and real-time EMSs based on model predictive control (MPC). Simulation results indicate that without priori knowledge of future trip, original DRL-based EMS achieves an average 3.5% gap from benchmark, superior to MPC-based EMS with accurate prediction; after further applying output frequency adjustment, a mean gap of 8.7%, which is comparable with MPC-based EMS with mean prediction error of 1m/s, is maintained with concurrently noteworthy improvement in reducing engine start times. Besides, its impressive computation speed of about 0.001s per simulation step proves its practical application potential, and this method is independent of powertrain topology such that it is applicative for any type of HEVs even when future driving information is unavailable.

This study aims to answer the key question for hybrid electric vehicles (HEVs) on how to manage the power flow in HEVs with recent intelligent and electrified upgrades in automotive industries. The new energy management strategy (EMS) needs to fuse both the physic and cyber systems, reflecting the dynamic vehicle system in the physical layer, as well as taking full advantage of the outside information in the cyber layer. Given that, this article proposes the cyber-physical system (CPS)-based EMS using deep reinforcement learning (DRL) in two different types of vehicles [hybrid electric bus (HEB) and Prius]. Under the proposed framework, exploratory training is carried out for the EMS which is mediated by DRL algorithms, expert prior knowledge and multistate of traffic information. Then, the prior valid knowledge trained by HEB is applied to Prius based on deep transfer learning, accelerating the new EMS convergence and ensuring the same initial parameters of the two vehicles' deep neural networks. Moreover, the cyber information is decoupled from the vehicle itself, for the first time being visualized for comparative analysis. The results show a significant improvement by considering traffic states (TS) and using dynamic programming (DP) as a benchmark with 6.94% fuel economy improvement for deep deterministic policy gradients (DDPG) test results and 8.12% for deep Q-learning (DQL), respectively. The decoupling analysis distinguished the effect of various TS for the HEB and Prius due to their different characteristics in vehicle service, driving style, and vehicle structures, which further verifies the effectiveness of the proposed EMS.

The hybrid-electric powertrain presents an immediate solution to energy and environmental challenges encountered within the realm of transportation. Targeting the optimization of hybrid-electric powertrains, deep reinforcement learning (DRL) has been intensively and increasingly investigated to develop intelligent energy management systems in the context of augmented vehicular and traffic information. After a brief introduction to the Markov Decision Process and DRL, this paper presents a comprehensive survey of the recent advancements in DRL-based energy management. The survey categorizes the progress based on the various roles that DRL plays in energy management systems, highlighting the flexibility and advantages of integrating DRL for achieving energy efficiency, safety, and reliable performance. Furthermore, the study concludes with an analysis of open issues and future prospects, including the learning and application of DRL-based energy management strategies, development of novel DRL algorithms, and integration of DRL-based energy management in intelligent and sustainable transportation contexts.

To combat the impacts of uncertain noise on the estimation of vehicle state parameters and the high cost of sensors, a state-observer design with an adaptive unscented Kalman filter (AUKF) is developed. The design equation of the state observer is derived by establishing the vehicle’s three degrees-of-freedom (DOF) model. On this basis, the Sage–Husa algorithm and unscented Kalman filter (UKF) are combined to form the AUKF algorithm to adaptively update the statistical feature estimation of measurement noise. Finally, a co-simulation using Carsim and Matlab/Simulink confirms the algorithm is effective and reasonable. The simulation results demonstrate that the proposed algorithm, compared with the UKF algorithm, increases estimation accuracy by 19.13%, 32.8%, and 39.46% in yaw rate, side-slip angle, and longitudinal velocity, respectively. This is because the proposed algorithm adaptively adjusts the measurement noise covariance matrix, which can estimate the state parameters of the vehicle more accurately.