7 April 2021 in IEEE Transactions on Industrial Electronics
Fast charging is an enabling technique for the large-scale penetration of electric vehicles. This article proposes a knowledge-based, multiphysics-constrained fast charging strategy for lithium-ion battery (LIB), with a consciousness of the thermal safety and degradation. A universal algorithmic framework combining model-based state observer and a deep reinforcement learning (DRL)-based optimizer is proposed, for the first time, to provide a LIB fast charging solution. Within the DRL framework, a multiobjective optimization problem is formulated by penalizing the over-temperature and degradation. An improved environmental perceptive deep deterministic policy gradient (DDPG) algorithm with priority experience replay is exploited to tradeoff smartly the charging rapidity and the compliance of physical constraints. The proposed DDPG-DRL strategy is compared experimentally with the rule-based strategies and the state-of-the-art model predictive controller to validate its superiority in terms of charging rapidity, enforcement of LIB thermal safety and life extension, as well as the computational tractability.
19 June 2023 in IEEE Transactions on Neural Networks and Learning Systems
Predicting the future states of surrounding traffic participants and planning a safe, smooth, and socially compliant trajectory accordingly are crucial for autonomous vehicles (AVs). There are two major issues with the current autonomous driving system: the prediction module is often separated from the planning module, and the cost function for planning is hard to specify and tune. To tackle these issues, we propose a differentiable integrated prediction and planning (DIPP) framework that can also learn the cost function from data. Specifically, our framework uses a differentiable nonlinear optimizer as the motion planner, which takes as input the predicted trajectories of surrounding agents given by the neural network and optimizes the trajectory for the AV, enabling all operations to be differentiable, including the cost function weights. The proposed framework is trained on a large-scale real-world driving dataset to imitate human driving trajectories in the entire driving scene and validated in both open-loop and closed-loop manners. The open-loop testing results reveal that the proposed method outperforms the baseline methods across a variety of metrics and delivers planning-centric prediction results, allowing the planning module to output trajectories close to those of human drivers. In closed-loop testing, the proposed method outperforms various baseline methods, showing the ability to handle complex urban driving scenarios and robustness against the distributional shift. Importantly, we find that joint training of planning and prediction modules achieves better performance than planning with a separate trained prediction module in both open-loop and closed-loop tests. Moreover, the ablation study indicates that the learnable components in the framework are essential to ensure planning stability and performance. Code and Supplementary Videos are available at https://mczhi.github.io/DIPP/ .
21 September 2020 in IEEE Transactions on Vehicular Technology
Energy management is an enabling technique to guarantee the reliability and economy of hybrid electric systems. This paper proposes a novel machine learning-based energy management strategy for a hybrid electric bus (HEB), with an emphasized consciousness of both thermal safety and degradation of the onboard lithium-ion battery (LIB) system. Firstly, the deep deterministic policy gradient (DDPG) algorithm is combined with an expert-assistance system, for the first time, to enhance the “cold start” performance and optimize the power allocation of HEB. Secondly, in the framework of the proposed algorithm, the penalties to over-temperature and LIB degradation are embedded to improve the management quality in terms of the thermal safety enforcement and overall driving cost reduction. The proposed strategy is tested under different road missions to validate its superiority over state-of-the-art techniques in terms of training efficiency and optimization performance.
21 June 2022 in IEEE Transactions on Intelligent Vehicles
To further improve learning efficiency and performance of reinforcement learning (RL), a novel uncertainty-aware model-based RL method is proposed and validated in autonomous driving scenarios in this paper. First, an action-conditioned ensemble model with the capability of uncertainty assessment is established as the environment model. Then, a novel uncertainty-aware model-based RL method is developed based on the adaptive truncation approach, providing virtual interactions between the agent and environment model, and improving RL’s learning efficiency and performance. The proposed method is then implemented in end-to-end autonomous vehicle control tasks, validated and compared with state-of-the-art methods under various driving scenarios. Validation results suggest that the proposed method outperforms the model-free RL approach with respect to learning efficiency, and model-based approach with respect to both efficiency and performance, demonstrating its feasibility and effectiveness.
16 March 2023 in IEEE Transactions on Intelligent Transportation Systems
Making safe and human-like decisions is an essential capability of autonomous driving systems, and learning-based behavior planning presents a promising pathway toward achieving this objective. Distinguished from existing learning-based methods that directly output decisions, this work introduces a predictive behavior planning framework that learns to predict and evaluate from human driving data. This framework consists of three components: a behavior generation module that produces a diverse set of candidate behaviors in the form of trajectory proposals, a conditional motion prediction network that predicts future trajectories of other agents based on each proposal, and a scoring module that evaluates the candidate plans using maximum entropy inverse reinforcement learning (IRL). We validate the proposed framework on a large-scale real-world urban driving dataset through comprehensive experiments. The results show that the conditional prediction model can predict distinct and reasonable future trajectories given different trajectory proposals and the IRL-based scoring module can select plans that are close to human driving. The proposed framework outperforms other baseline methods in terms of similarity to human driving trajectories. Additionally, we find that the conditional prediction model improves both prediction and planning performance compared to the non-conditional model. Lastly, we note that the learning of the scoring module is crucial for aligning the evaluations with human drivers.
29 April 2021 in IEEE Transactions on Vehicular Technology
With the aims of safe, smart and sustainable future mobility, a personalized approach of trajectory planning and control based on user preferences is developed for lane-change of autonomous vehicles in this paper. First, a safe area during the lane change process is identified by using constraint Delaunay triangulation. Then, an improved rapidly-exploring Random Trees (i-RRT) is developed with B-spline to generate the feasible trajectory cluster, which is subject to the safe area boundaries and the vehicle dynamics. To extract a personalized trajectory from this cluster, we firstly adopt the fuzzy linguistic preference relation (FLPR) method to identify users' preferences on driving, which can be reflected by their subjective objectives including driving safety, ride comfort and vehicle stability. Then, the technique for order preference by similarity to ideal situation (TOPSIS) is utilized to solve the multi-objective optimisation problem formulated by considering the user preferences. The algorithms proposed above are integrated, and both simulation and experimental validation are conducted under lane-change scenarios of autonomous driving. Simulation and experiment results show that proposed approach is able to successfully realize personalized trajectory planning and lane-change control, satisfying users' various preferences and simultaneously ensure vehicle safety, demonstrating its feasibility and effectiveness.