Reinforcement learning (RL) based energy management is one of the current hot-spots of hybrid electric vehicles. Recent advances in RL based energy management focus on energy-saving performance but less considers the constrained setting for training safety. This paper proposes an RL framework named Coach-Actor-Double Critic (CADC) for the optimization of energy management considered as the constrained Markov Decision Process (CMDP). A bi-level onboard controller includes a neural network (NN) based strategy Actor and rule-based strategy Coach for online energy management. Once the output of Actor exceeds the constrained range of feasible solutions, Coach would take charge of energy management to ensure safety. By using Lagrangian relaxation, the optimization for constrained Markov decision process transforms into an unconstrained dual problem to minimize the energy consumption while minimizing the Coach participation. The parameters of Actor are updated in a manner of policy gradient through RL training with the Lagrangian value function. Double Critic with the same structure synchronously estimate the value function to avoid overestimate bias. Several experiments with the bus trajectories data demonstrate the optimality, self-learning ability, and adaptability of CADC. The results indicate that CADC outperforms the existing RL based strategies and reaches above 95% energy-saving rate of the offline global optimum.

Coordinating a platoon of connected hybrid electric vehicles (HEVs) poses challenges due to the intricacy of their powertrains and the diverse driving scenarios encountered. The existing mainstream framework uses a hierarchical control scheme, simplifying the unified optimization problem into two separate series control processes: the powertrain level and the vehicle level. However, this approach overlooks the inherent interdependence between the vehicle and powertrain systems, which can hinder effective optimization and collaboration in terms of energy management across multiple vehicles. To address this problem, a multi-agent reinforcement learning-based energy control framework is proposed, aiming to unleash the energy-saving potential through an integrated collaborative optimization of velocity optimization and energy management strategy for HEV platoon. The proposed strategy constructs a joint-goals value function based on Markov games for HEV platooning and utilizes long short-term memory networks to capture temporal associations of the platoon dynamics. In addition, an asynchronous reinforcement learning method is introduced for knowledge sharing among HEVs in the platoon. The simulation results demonstrate that the proposed approach effectively improves driving behavior and powertrain energy efficiency through multi-vehicle coordination. Compared to the rule-based baseline, the fuel consumption of the platoon is reduced by 19.2% through the coordination of connected HEVs.

Understanding the underlying patterns of the urban mobility dynamics is essential for both the traffic state estimation and management of urban facilities and services. Due to the coupling relationship of generative factors in spatial-temporal domain, it is challenging to model the citywide traffic dynamics under a structural pattern of critical features such as hours of days, days of weeks and weather conditions. To address this challenge, this article develops a disentangled representation learning framework to learn an interpretable factorized representation of the independent data generative factors. In order to make full use of the knowledge on generative factors, this article proposes spatial-temporal generative adversarial network (ST-GAN) to assign the generative factors of traffic flow to the feature vector in latent space and reconstructs the high-dimensional citywide traffic flow from the given factors. With the help of the disentangled representations, the decomposed feature vector in latent space discloses the relationship between underlying patterns and citywide traffic dynamics. Several comprehensively experiments show that ST-GAN not only effectively improves the prediction accuracy but also promisingly characterize structural properties of the traffic evolution process.

Cooperative control of connected and automated electric vehicles (CAEVs) offers a great potential for a safe, high-efficient, sustainable transportation system. Among them, coordinated on-ramp merging based on an on-ramp merging vehicle and a mainline facilitating is a hot spot to lighten the impact of shockwave on highway junctions. Reinforcement learning (RL) is a promising solution to address this problem for its strong adaptiveness and self-learning ability. The existing methods are dedicatedly designed to circumvent the sparse reward problem of long-term merging safety but restrict the optimization space and neglect individual costs, especially irreversible loss, such as battery aging. In this article, a hindsight experience replay RL coordinated freeway on-ramp merging frame is proposed to face the sparse reward problem in ramp merging problem and, meanwhile, co-optimize traffic efficiency, energy-saving, and battery health performance. Compared with the existing optimization-based strategy, it fundamentally avoids abundant expert knowledge and elaborate design for reward shaping and model construction while ensuring the nature of merging tasks without excessive simplification. Numerical experiments are conducted to verify the optimality, adaptability, and self-learning ability of the proposed strategy. With comparison experiment, the proposed strategy surpasses state-of-the-art RL methods more than 13% in overall index throughput with 12% energy consumption while easing individual battery aging during the merging process.

Whether the powertrain parameters are reasonable will directly affect the fuel economy of Plug-in hybrid electric bus (PHEB). In this paper, the fuel economy is chosen as the optimization target of a serial-parallel PHEB. A global optimal strategy, which is formulated by dynamic programming (DP) algorithm, is used as energy management strategy for PHEB. Firstly, PHEB fuel economy is chosen as the optimization objective. Then, a combinatorial optimization algorithm is designed by combining Multi-Island genetic algorithm (MIGA) with NLPQL. MIGA is used for global optimization, and the NLPQL is used for local optimization to make up for poor ability of MIGA in local optimization. Finally, several hardware in loop (HIL) experiments were carried out and the results prove that the fuel consumption per 100 km has reduced from 25.7 l diesel to 22.9 l diesel, and the electricity consumption per 100 km has reduced from 14.7kW∙h to 14.3kW∙h.