Sunan Zhang

Sunan Zhang

Auto-generated profile, based on publicly available information

Sunan Zhang

School of Mechanical Engineering, Southeast University, Nanjing, China

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
24 April 2024 in IEEE Transactions on Transportation Electrification
Verified
Journal Article
Closed access
SZSunan Zhang
Sunan Zhang
Weichao ZhuangWeichao Zhuang
Weichao Zhuang
BLBingbing Li
Bingbing Li
KLKe Li
Ke Li
TXTianyu Xia
Tianyu Xia
BHBo Hu
Bo Hu
The intricate interactions with other road users and the diversity of traffic environments create a challenging decision-making task for autonomous driving systems. While offline learning solutions are renowned for their high execution efficiency and the ability to approximate the optimal policy across the entire state space, they are often unsafe and fragile when encountering untrained states. Conversely, online planning methods possess the capacity to thoroughly assess how current decisions influence future outcomes online, and therefore have better generalization. Nevertheless, these approaches face difficulties in terms of execution efficiency and are susceptible to becoming stuck in locally optimal solutions. In this context, this paper proposes an Integrated Planning and Learning (IPL) algorithm based on the reinforcement learning framework for speed and lane change decision-making on highways. Specifically, at each decision time step, this method utilizes the offline learned model to guide an online Monte Carlo Tree Search (MCTS) algorithm for heuristic search, aiming to formulate a forward-looking policy. The experimental results show that the IPL algorithm performs better generalization when faced with unknown scenarios, and its asymptotic performance is better than other benchmark algorithms. In addition, in contrast to the MCTS-based online planning method, the IPL algorithm enhances execution efficiency and comes closer to achieving global optimality.

17 April 2023 in IEEE Transactions on Transportation Electrification
Verified
Journal Article
Closed access
BHBo Hu
Bo Hu
SZSunan Zhang
Sunan Zhang
BLBocheng Liu
Bocheng Liu
Energy management strategy (EMS) is one of the key technologies that improves the fuel efficiency of hybrid electric vehicles (HEVs) by governing the energy flow between the fuel tank and the electric energy storage. With the rapid development of artificial intelligence especially after the great success of AlphaGo, reinforcement learning (RL) has opened up a new window for EMS. Although many RL-based solutions have been successfully applied to EMS tasks, most current RL-based approaches only consider RL as an offline optimization tool, i.e., RL is used to solve optimization problems given a simulation model. This method may be utilized to realize optimal control in simulation, but there is no guarantee that a well-behaved performance can also be achieved in the real-physical world due to the simulation-to-real gap. Meanwhile, for many industrial EMS tasks especially when optimizing an existing controller, a high-fidelity simulator may be unavailable, but some logging data generated by some suboptimal existing controller and simple models of the principal powertrain components neglecting detailed component dynamics could exist. In this context, a hybrid algorithm combining data-driven and simulation-based RL approach is proposed that tries to learn experience from both the real logging data and simulated simple model. Based on the hardware-in-the-loop (HIL) results, a near optimal policy can be obtained (fuel consumption is decreased by approximately 6.10%) by this hybrid algorithm from a small batch logging data and a simple simulated model when compared with the dynamic programming (DP) method using a high-fidelity simulation model mimicking the real-physical world.

25 December 2023 in IEEE Transactions on Transportation Electrification
Verified
Journal Article
Closed access
BHBo Hu
Bo Hu
LJLei Jiang
Lei Jiang
SZSunan Zhang
Sunan Zhang
Qiang WangQiang Wang
Qiang Wang
Reinforcement learning (RL) has the capability to discover optimal interactions with the surrounding environment, with the advantage that nearly all required computations can be performed offline. Nevertheless, the lack of explainability for RL-based solutions may prevent their large-scale application in industrial autonomous vehicle tasks. Furthermore, the RL method tends to be unsafe and brittle to scenarios not encountered in training. Conversely, the optimization-based method offers a substantial level of explainability, and through the explicit inclusion of safety constraints, it can guarantee the system’s safety. In this context, building upon RL framework, a fusion algorithm that combines the advantage of the RL-based scheme and optimization-based scheme is proposed. Specifically, unlike traditional RL-based solutions which directly executes from perception to control using only neural network maps, this work introduces a mechanism of uncertainty-aware interval prediction to compute the set of states that can be reached over the planning time horizon. On this basis, a robust control framework is presented, which guarantees system safety while considering the worst-case performance scenarios. To validate the proposed algorithm, the task of an autonomous vehicle merging on to a highway from an on-ramp is simulated in SUMO. The results show that the proposed motion planning and control method combines the advantages of RL and optimization-based methods and achieves balanced performance in smoothness, computational efficiency, explainability and robustness.

6 February 2024 in IEEE Transactions on Industrial Electronics
Verified
Journal Article
Closed access
BHBo Hu
Bo Hu
BLBocheng Liu
Bocheng Liu
SZSunan Zhang
Sunan Zhang
Considering the simulation-to-real gap and the fact that the data-driven learning methods are often suboptimal, an effective offline-to-online learning paradigm that can further improve the initialized offline policy in a real environment is necessary. However, for most industrial applications, this two-stage learning paradigm is often challenging to implement, as the initialized policy frequently deviates from the optimal improvement during fine-tuning. To address this, an effective offline-to-online reinforcement learning (RL) based training framework that bridges offline initialization and online fine-tuning will be proposed in this work. In contrast to most of RL algorithms, which evaluate unseen actions from the latest policy and extract policies only corresponding to the maximum value function, the proposed method uses two implicit constraints, namely, quantile regression and advantage weighted regression. In order to assess the efficacy of the proposed algorithm with real controllers, a hardware-in-the-loop test is conducted. The findings demonstrate that by leveraging the generalization capability of the quantile regression to estimate the value of the best available state-action pair and derive the policy using the advantage-weighted form of behavioral cloning to favor actions that receive higher advantage, the proposed data-driven RL can not only learn effectively from a static offline dataset, but also exhibit a robust policy improvement during the subsequent fine-tuning process.

2 September 2025 in Engineering Applications of Artificial Intelligence
Verified
Journal Article
Closed access
PLPing Lu
Ping Lu
SZSunan Zhang
Sunan Zhang
FTFeihong Tan
Feihong Tan
FZFulin Zhang
Fulin Zhang
YFYuxiang Feng
Yuxiang Feng
BHBo Hu
Bo Hu

2 November 2024 in Engineering Applications of Artificial Intelligence
Verified
Journal Article
Closed access
BHBo Hu
Bo Hu
SZSunan Zhang
Sunan Zhang
YFYuxiang Feng
Yuxiang Feng
BLBingbing Li
Bingbing Li
HSHao Sun
Hao Sun
YZYi Zhang
Yi Zhang