The energy management control strategy of hybrid electric vehicles is a critical research field. During driving, a vehicle encounters various uncertain and even intricate driving cycles for a vehicle. Although reinforcement learning has clear advantages in energy management control strategies, fuel consumption remains high in the driving cycles. To solve these problems, this study proposes an uncertainty-based reinforcement learning method, which uses a distributed uncertainty function to select the action when faced with intricate regions of driving cycles. The algorithm implementation processes are as follows: First, uncertainty is calculated by the uncertainty module; Then, the action values, in situations of low uncertainty, are derived from the original reinforcement learning algorithm, while in situations of high uncertainty, they are obtained from the uncertainty module. Results indicate that through the application of uncertainty-based reinforcement learning method, the fuel economy is improved by about 5% compared to the Deep Deterministic Policy Gradient (DDPG) algorithm and about 11% higher than the Deep Q-Network (DQN) algorithm under intricate driving cycles.

Energy management strategies typically employ reinforcement learning algorithms in a static state. However, during vehicle operation, the environment is dynamic and laden with uncertainties and unforeseen disruptions. This study proposes an adaptive learning strategy in dynamic environments that adapts actions to changing circumstances, drawing on past experience to enhance future real-world learning. We developed a memory library for dynamic environments, employed Dirichlet clustering for driving conditions, and incorporated the expectation maximization algorithm for timely model updating to fully absorb prior knowledge. The agent swiftly adapts to the dynamic environment and converges quickly, improving hybrid electric vehicle fuel economy by 5–10% while maintaining the final state of charge (SOC). Our algorithm’s engine operating point fluctuates less, and the working state is compact compared with Deep Q-Network (DQN) and Deterministic Policy Gradient (DDPG) algorithms. This study provides a solution for vehicle agents in dynamic environmental conditions, enabling them to logically evaluate past experiences and carry out situationally appropriate actions.