Hongpeng Zhang

Dr. Hongpeng Zhang

orcid

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
11 October 2022 in Applied Sciences
Verified
Journal Article
Open access
PDF file
HZHongpeng Zhang
Hongpeng Zhang
YWYujie Wei
Yujie Wei
HZHuan Zhou
Huan Zhou
CHChangqiang Huang
Changqiang Huang
Maneuver decision-making is the core of autonomous air combat, and reinforcement learning is a potential and ideal approach for addressing decision-making problems. However, when reinforcement learning is used for maneuver decision-making for autonomous air combat, it often suffers from awful training efficiency and poor performance of maneuver decision-making. In this paper, an air combat maneuver decision-making method based on final reward estimation and proximal policy optimization is proposed to solve the above problems. First, an air combat environment based on aircraft and missile models is constructed, and an intermediate reward and final reward are designed. Second, the final reward estimation is proposed to replace the original advantage estimation function of the surrogate objective of proximal policy optimization to improve the training performance of reinforcement learning. Third, sampling according to the final reward estimation is proposed to improve the training efficiency. Finally, the proposed method is used in a self-play framework to train agents for maneuver decision-making. Simulations show that final reward estimation and sampling according to final reward estimation are effective and efficient.

19 August 2023 in Applied Sciences
Verified
Journal Article
Open access
PDF file
YWYujie Wei
Yujie Wei
HZHongpeng Zhang
Hongpeng Zhang
YWYuan Wang
Yuan Wang
CHChangqiang Huang
Changqiang Huang
Maneuver decision-making is essential for autonomous air combat. However, previous methods usually make decisions to aim at the target instead of hitting the target and use discrete action spaces instead of continuous action spaces. While these simplifications make maneuver decision-making easier, they also make maneuver decision-making more unrealistic. Meanwhile, previous studies usually rely on handcrafted reward functions, which are troublesome to design. Therefore, to solve these problems, we propose an automatic curriculum reinforcement learning method that enables agents to maneuver effectively in air combat from scratch. On the basis of curriculum reinforcement learning, maneuver decision-making is divided into a series of sub-tasks from easy to difficult. Thus, agents can gradually learn how to complete a series of sub-tasks, from easy to difficult without handcrafted reward functions. The ablation studies show that automatic curriculum learning is essential for reinforcement learning; namely, agents cannot make effective decisions without curriculum learning. Simulations show that, after training, agents are able to make effective decisions given different states, including tracking, attacking, and escaping, which are both rational and interpretable.