Energy management strategy (EMS) is critical for improving the economy of hybrid powertrains and the durability of energy sources. In this paper, a novel EMS based on a twin delayed deep deterministic policy gradient algorithm (TD3) is proposed for a fuel cell hybrid electric bus (FCHEB) to optimize the driving cost of the vehicle. First, a TD3-based energy management strategy is established to embed the limits of battery aging and fuel cell power variation into the strategic framework to fully exploit the economic potential of FCHEB. Second, the TD3-based EMS is compared and analyzed with the deep deterministic policy gradient algorithm (DDPG)-based EMS using real-world collected driving conditions as training data. The results show that the TD3-based EMS has 54.69% higher training efficiency, 36.82% higher learning ability, and 2.45% lower overall vehicle operating cost compared to the DDPG-based EMS, validating the effectiveness of the proposed strategy.