Zhen Liu

Zhen Liu


Research Focus & Metrics

Academic Career Length

9Years (2018-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
10 October 2019 in IEEE Transactions on Industrial Informatics
Verified
Journal Article
Closed access
JSJinlin Sun
Jinlin Sun
ZPZhiqiang Pu
Zhiqiang Pu
JYJianqiang Yi
Jianqiang Yi
ZLZhen Liu
Zhen Liu
Uncertainty and measurement noise are main obstacles that limit the tracking control performances of flexible air-breathing hypersonic vehicles (FAHVs). In this paper, we propose a novel fixed-time convergent nonsmooth backstepping control scheme for FAHV via augmented sliding mode observers (ASMOs) to overcome these obstacles. The ASMOs are first designed for the FAHV dynamics by employing the measured states corrupted by noises as inputs. On one hand, the ASMOs can simultaneously estimate the uncertainties and filter out the measurement noises. On the other hand, the observation error of each ASMO can be convergent within a fixed time independent of its initial observation error. Then, based on the estimation results, the altitude and velocity tracking controllers are developed by using fixed-time nonsmooth backstepping technique. Afterwards, a Lyapunov-based stability analysis is given to illustrate the fixed-time convergence of the closed-loop signals of the FAHV control system. Finally, comparative simulations are conducted to illustrate the superiority of the proposed control scheme.

18 February 2022 in IEEE Transactions on Neural Networks and Learning Systems
Verified
Journal Article
Closed access
ZPZhiqiang Pu
Zhiqiang Pu
HWHuimu Wang
Huimu Wang
ZLZhen Liu
Zhen Liu
JYJianqiang Yi
Jianqiang Yi
SWShiguang Wu
Shiguang Wu
In this article, a novel method, called attention enhanced reinforcement learning (AERL), is proposed to address issues including complex interaction, limited communication range, and time-varying communication topology for multi agent cooperation. AERL includes a communication enhanced network (CEN), a graph spatiotemporal long short-term memory network (GST-LSTM), and parameters sharing multi-pseudo critic proximal policy optimization (PS-MPC-PPO). Specifically, CEN based on graph attention mechanism is designed to enlarge the agents’ communication range and to deal with complex interaction among the agents. GST-LSTM, which replaces the standard fully connected (FC) operator in LSTM with graph attention operator, is designed to capture the temporal dependence while maintaining the spatial structure learned by CEN. PS-MPC-PPO, which extends proximal policy optimization (PPO) in multi agent systems with parameters’ sharing to scale to environments with a large number of agents in training, is designed with multi-pseudo critics to mitigate the bias problem in training and accelerate the convergence process. Simulation results for three groups of representative scenarios including formation control, group containment, and predator–prey games demonstrate the effectiveness and robustness of AERL.

12 August 2019 in IEEE Transactions on Fuzzy Systems
Verified
Journal Article
Closed access
JSJinlin Sun
Jinlin Sun
JYJianqiang Yi
Jianqiang Yi
ZPZhiqiang Pu
Zhiqiang Pu
ZLZhen Liu
Zhen Liu
It is commonly believed that uncertainties are obstacles to the tracking performances of flexible air-breathing hypersonic vehicles (FAHVs). In addition, the estimation of unmeasured states in FAHV makes this issue much more complicated. To deal with these difficulties, this paper explores a novel adaptive fuzzy nonsmooth backstepping output-feedback control scheme for FAHV with closed-loop finite-time convergence. Firstly, to approximate the unknown dynamics of FAHV, some function approximators are constructed by utilizing interval type-2 fuzzy logic system (IT2-FLS). On this basis, a fixed-time convergent adaptive IT2 fuzzy observer is designed to estimate the unmeasured flight path angle and angle of attack of FAHV accurately and rapidly. Consequently, the estimation errors of the designed observer are convergent in a fixed time independent of their initial estimation errors. Furthermore, based on the estimated states, velocity and altitude tracking controllers are designed by using finite-time adaptive IT2 fuzzy nonsmooth backstepping control technique, which avoids the problem of “explosion of complexity” in traditional backstepping methods. Subsequently, rigorous Lyapunov stability analysis is conducted to show the finite-time convergence of the closed-loop signals of FAHV control system. Finally, the robustness and superiority of the proposed control scheme is validated by several simulations in representative scenarios.

14 October 2022 in IEEE Transactions on Emerging Topics in Computational Intelligence
Verified
Journal Article
Closed access
TZTianle Zhang
Tianle Zhang
ZLZhen Liu
Zhen Liu
ZPZhiqiang Pu
Zhiqiang Pu
JYJianqiang Yi
Jianqiang Yi
Recently, a lot of works have been devoted to researching how agents can learn efficient cooperation in multiagent systems. However, it still remains challenging in large-scale multiagent systems (MASs) due to the complex dynamics between the agents and environment and the dimension explosion of state-action space. In this paper, we propose a novel MultiAgent Automatic Curriculum Learning method (MA-ACL) to solve learning problems of large-scale cooperative MASs by beginning from learning on a multiagent scenario with a few agents and automatically progressively increasing the number of agents. An evaluation mechanism based on self-supervised learning is innovatively designed to automatically generate appropriate curricula with a progressively increasing number of agents. Moreover, since the observation state dimension of agents varies across curricula and the learned policy knowledge needs to be effectively encoded, we design a new Distributed Transferable Relation-modeling Policy network structure (DTRP) to handle the dynamic size of the network input and model relational knowledge between agents and their surrounding environment. Simulation results show that the proposed MA-ACL using DTRP can significantly improve the performance of large-scale multiagent learning compared with manual or non curriculum learning methods, and DTRP greatly boosts the performance of MA-ACL.

8 August 2022 in IEEE Transactions on Games
Verified
Journal Article
Closed access
TZTianle Zhang
Tianle Zhang
ZLZhen Liu
Zhen Liu
ZPZhiqiang Pu
Zhiqiang Pu
JYJianqiang Yi
Jianqiang Yi
Social learning, especially social incentives, is extremely important for humans to achieve a high level of coordination. Inspired by this, we introduce this concept into cooperative multi-agent reinforcement learning (MARL), to implicitly address the credit assignment problem and promote the inter-agent direct interactions for cooperations among agents in cooperative multi-agent games. In this paper, we propose a novel Intrinsic Reward method with Peer Incentives (IRPI) based on actor-critic policy gradient. This method can enable agents to incentivize each other for their cooperations through using causal influence among them. Specifically, a novel intrinsic reward mechanism is innovatively designed to empower each agent the ability to give positive or negative rewards to other peer agents' actions through considering the causal influence of the other agents on it. The mechanism is realized by a feed-forward neural network through utilizing causal influence between the agents. The causal influence of one agent on another is inferred via counterfactual reasoning using the joint action-value function in MARL. The quality of the influence is assessed via counterfactual reasoning using the individual value function in MARL. Simulations are carried out on two popular multi-agent game testbeds: Starcraft II Micromanagement and Multi-Agent Particle Environments. Simulational results demonstrate that the proposed IRPI can enhance cooperations among the agents to achieve better performance compared with a number of state-of-the-art MARL methods in a variety of cooperative multi-agent games.

5 May 2023 in International Journal of Control, Automation, and Systems
Verified
Journal Article
Closed access
PDF file
TZTianle Zhang
Tianle Zhang
ZLZhen Liu
Zhen Liu
ZPZhiqiang Pu
Zhiqiang Pu
JYJianqiang Yi
Jianqiang Yi
YLYanyan Liang
Yanyan Liang
DZDu Zhang
Du Zhang