Yichuan Zhang

Yichuan Zhang

Auto-generated profile, based on publicly available information

Yichuan Zhang

Xian Satellite Control Center, Xian, China

Research Focus & Metrics

Academic Career Length

6Years (2021-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
20 September 2023 in IEEE Transactions on Aerospace and Electronic Systems
Verified
Journal Article
Closed access
JLJun Liang
Jun Liang
JLJian-Ping Liu
Jian-Ping Liu
QSQing Sun
Qing Sun
YZYue-He Zhu
Yue-He Zhu
YZYi-Chuan Zhang
Yi-Chuan Zhang
BHBo-Yong He
Bo-Yong He
Real-time emergency event processing for mega-constellation management requires an approach that can quickly obtain a satisfactory satellite range rescheduling scheme. In this paper, a deep reinforcement learning (DRL) approach is proposed for rapid rescheduling on the basis of established satellite range scheduling schemes. Three rescheduling principles with different impacts on established scheduling schemes are first analyzed. A mathematical model for both maximizing the completion profits of dynamic missions and minimizing their impacts on scheduled missions is formulated. An efficient DRL-based approach to satellite range rescheduling is then developed. A mask processing mechanism is integrated into the reinforcement learning framework to effectively eliminate the conflicts accurately expressed by the conflict space, and a modified pointer network (Ptr-Net) based on the mask processing mechanism is employed to approximate the action-value function for optimal decision-making. Experimental results show that the proposed approach significantly outperforms the rule-based heuristic and another DRL-based method in terms of computational efficiency and optimization performance, and the total time spent during the training and testing phases of the proposed approach is approximately 13% of the total running time required by the search-based methods.

1 January 2021 in Computational Intelligence and Neuroscience
Verified
Journal Article
Open access
PDF file
YZYichuan Zhang
Yichuan Zhang
YLYixing Lan
Yixing Lan
QFqiang fang
qiang fang
XXXin Xu
Xin Xu
JLJunxiang Li
Junxiang Li
YZYujun Zeng
Yujun Zeng
Reinforcement learning from demonstration (RLfD) is considered to be a promising approach to improve reinforcement learning (RL) by leveraging expert demonstrations as the additional decision‐making guidance. However, most existing RLfD methods only regard demonstrations as low‐level knowledge instances under a certain task. Demonstrations are generally used to either provide additional rewards or pretrain the neural network‐based RL policy in a supervised manner, usually resulting in poor generalization capability and weak robustness performance. Considering that human knowledge is not only interpretable but also suitable for generalization, we propose to exploit the potential of demonstrations by extracting knowledge from them via Bayesian networks and develop a novel RLfD method called Reinforcement Learning from demonstration via Bayesian Network‐based Knowledge (RLBNK). The proposed RLBNK method takes advantage of node influence with the Wasserstein distance metric (NIW) algorithm to obtain abstract concepts from demonstrations and then a Bayesian network conducts knowledge learning and inference based on the abstract data set, which will yield the coarse policy with corresponding confidence. Once the coarse policy’s confidence is low, another RL‐based refine module will further optimize and fine‐tune the policy to form a (near) optimal hybrid policy. Experimental results show that the proposed RLBNK method improves the learning efficiency of corresponding baseline RL algorithms under both normal and sparse reward settings. Furthermore, we demonstrate that our RLBNK method delivers better generalization capability and robustness than baseline methods.

16 March 2023 in CAAI Transactions on Intelligence Technology
Verified
Journal Article
Open access
JRJunkai Ren
Junkai Ren
YLYixing Lan
Yixing Lan
XXXin Xu
Xin Xu
YZYichuan Zhang
Yichuan Zhang
QFQiang Fang
Qiang Fang
YZYujun Zeng
Yujun Zeng
Policy evaluation (PE) is a critical sub‐problem in reinforcement learning, which estimates the value function for a given policy and can be used for policy improvement. However, there still exist some limitations in current PE methods, such as low sample efficiency and local convergence, especially on complex tasks. In this study, a novel PE algorithm called Least‐Squares Truncated Temporal‐Difference learning (LST 2 D) is proposed. In LST 2 D, an adaptive truncation mechanism is designed, which effectively takes advantage of the fast convergence property of Least‐Squares Temporal Difference learning and the asymptotic convergence property of Temporal Difference learning (TD). Then, two feature pre‐training methods are utilised to improve the approximation ability of LST 2 D. Furthermore, an Actor‐Critic algorithm based on LST 2 D and pre‐trained feature representations (ACLPF) is proposed, where LST 2 D is integrated into the critic network to improve learning‐prediction efficiency. Comprehensive simulation studies were conducted on four robotic tasks, and the corresponding results illustrate the effectiveness of LST 2 D. The proposed ACLPF algorithm outperformed DQN, ACER and PPO in terms of sample efficiency and stability, which demonstrated that LST 2 D can be applied to online learning control problems by incorporating it into the actor‐critic architecture.

9 August 2021
Verified
Book Chapter
Closed access
PDF file
YZYichuan Zhang
Yichuan Zhang
JRJunkai Ren
Junkai Ren
JLJunxiang Li
Junxiang Li
QFqiang fang
qiang fang
XXXin Xu
Xin Xu