Weiren Kong

Weiren Kong


Research Focus & Metrics

Academic Career Length

8Years (2019-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
10 July 2020 in Electronics
Verified
Journal Article
Open access
PDF file
WKWeiren Kong
Weiren Kong
DZdeyun zhou
deyun zhou
ZYZhen Yang
Zhen Yang
YZYiyang Zhao
Yiyang Zhao
KZKai Zhang
Kai Zhang
With the development of unmanned aerial vehicle (UAV) and artificial intelligence (AI) technology, Intelligent UAV will be widely used in future autonomous aerial combat. Previous researches on autonomous aerial combat within visual range (WVR) have limitations due to simplifying assumptions, limited robustness, and ignoring sensor errors. In this paper, in order to consider the error of the aircraft sensors, we model the aerial combat WVR as a state-adversarial Markov decision process (SA-MDP), which introduce the small adversarial perturbations on state observations and these perturbations do not alter the environment directly, but can mislead the agent into making suboptimal decisions. Meanwhile, we propose a novel autonomous aerial combat maneuver strategy generation algorithm with high-performance and high-robustness based on state-adversarial deep deterministic policy gradient algorithm (SA-DDPG), which add a robustness regularizers related to an upper bound on performance loss at the actor-network. At the same time, a reward shaping method based on maximum entropy (MaxEnt) inverse reinforcement learning algorithm (IRL) is proposed to improve the aerial combat strategy generation algorithm’s efficiency. Finally, the efficiency of the aerial combat strategy generation algorithm and the performance and robustness of the resulting aerial combat strategy is verified by simulation experiments. Our main contributions are three-fold. First, to introduce the observation errors of UAV, we are modeling air combat as SA-MDP. Second, to make the strategy network of air combat maneuver more robust in the presence of observation errors, we introduce regularizers into the policy gradient. Third, to solve the problem that air combat’s reward function is too sparse, we use MaxEnt IRL to design a shaping reward to accelerate the convergence of SA-DDPG.

24 December 2022 in IET Control Theory & Applications
Verified
Journal Article
Open access
WKWeiren Kong
Weiren Kong
DZDe‐Yun Zhou
De‐Yun Zhou
YDYong‐Jie Du
Yong‐Jie Du
YZYing Zhou
Ying Zhou
YZYi‐Yang Zhao
Yi‐Yang Zhao
The close‐range autonomous air combat has gained significant attention from researchers involved in applications related to artificial intelligence (AI). A majority of the previous studies on autonomous air combat were focused on one‐on‐one air combat scenarios, however, the modern air combat is mostly conducted in formations. With regard to the aforementioned factors, a novel hierarchical maneuvering control architecture is introduced that is applied to the multi‐aircraft close‐range air combat scenario, which can handle air combat scenarios with variable‐size formation. Subsequently, three air combat sub‐tasks are designed, and recurrent soft actor‐critic (RSAC) algorithm combined with competitive self‐play (SP) is incorporated to learn the sub‐strategies. A novel hierarchical multi‐agent reinforcement learning (HMARL) algorithm is proposed to obtain the high‐level strategy for target and sub‐strategy selection. The training performance of the training algorithm of sub‐strategies and high‐level strategy in different air combat scenarios is evaluated. The obtained strategies are analyzed and it is found that the formations exhibit effective cooperative behavior in symmetric and asymmetric scenarios. Finally, the ideas of engineering implementation of the maneuvering control architecture are given. The study provides a solution for future multi‐aircraft autonomous air combat.

6 March 2020 in IEEE Access
Verified
Journal Article
Open access
ZYZhen Yang
Zhen Yang
DZdeyun zhou
deyun zhou
HPHaiyin Piao
Haiyin Piao
KZKai Zhang
Kai Zhang
WKWeiren Kong
Weiren Kong
QPQian Pan
Qian Pan
This study deals with the autonomous evasive maneuver strategy of unmanned combat air vehicle (UCAV), which is threatened by a high-performance beyond-visual-range (BVR) air-to-air missile (AAM). Considering tactical demands of achieving self-conflicting evasive objectives in actual air combat, including higher miss distance, less energy consumption and longer guidance support time, the evasive maneuver problem in BVR air combat is defined and reformulated into a multi-objective optimization problem. Effective maneuvers of UCAV used in different evasion phases are modeled in three-dimensional space. Then the three-level decision space structure is established according to qualitative evasive tactical planning. A hierarchical multi-objective evolutionary algorithm (HMOEA) is designed to find the approximate Pareto-optimal solutions of the problem. The approach combines qualitative tactical experience and quantitative maneuver decision optimization method effectively. Simulations are used to demonstrate the feasibility and effectiveness of the approach. The results show that the obtained set of decision variables constitutes nondominated solutions, which can meet different evasive tactical requirements of UCAV while ensuring successful evasion.

28 July 2020 in Applied Sciences
Verified
Journal Article
Open access
PDF file
WKWeiren Kong
Weiren Kong
DZdeyun zhou
deyun zhou
ZYZhen Yang
Zhen Yang
KZKai Zhang
Kai Zhang
LZLina Zeng
Lina Zeng
With the development of unmanned combat air vehicles (UCAVs) and artificial intelligence (AI), within visual range (WVR) air combat confrontations utilizing intelligent UCAVs are expected to be widely used in future air combats. As controlling highly dynamic and uncertain WVR air combats from the ground stations of the UCAV is not feasible, it is necessary to develop an algorithm that can generate highly intelligent air combat strategies in order to enable UCAV to independently complete air combat missions. In this paper, a 1-vs.-1 WVR air combat strategy generation algorithm is proposed using the multi-agent deep deterministic policy gradient (MADDPG). A 1-vs.-1 WVR air combat is modeled as a two-player zero-sum Markov game (ZSMG). A method for predicting the position of the target is introduced into the model in order to enable the UCAV to predict the target’s actions and position. Moreover, to ensure that the UCAV is not limited by the constraints of the basic fighter maneuver (BFM) library, the action space is considered to be a continuous one. At the same time, a potential-based reward shaping method is proposed in order to improve the efficiency of the air combat strategy generation algorithm. Finally, the efficiency of the air combat strategy generation algorithm and the intelligence level of the resulting strategy is verified through simulation experiments. The results show that an air combat strategy using target position prediction is superior to the one that does not use target position prediction.

15 September 2020 in Electronics
Verified
Journal Article
Open access
PDF file
KZKai Zhang
Kai Zhang
DZdeyun zhou
deyun zhou
ZYZhen Yang
Zhen Yang
YZYiyang Zhao
Yiyang Zhao
WKWeiren Kong
Weiren Kong
The weapon-target assignment problem is a crucial decision support in a Command and Control system. As a typical operational scenario, the major asset-based dynamic weapon target assignment (A-DWTA) models and solving algorithms are challenging to reflect the actual requirement of decision maker. Deriving from the “shoot–look–shoot” principle, an “observe–orient–decide–act” loop model for A-DWTA (OODA/A-DWTA) is established. Focus on the decide phase of the OODA/A-DWTA loop, a novel A-DWTA model, which is based on the receding horizon decomposition strategy (A-DWTA/RH), is established. To solve the A-DWTA/RH efficiently, a heuristic algorithm based on statistical marginal return (HA-SMR) is designed, which proposes a reverse hierarchical idea of “asset value-target selected-weapon decision.” Experimental results show that HA-SMR solving A-DWTA/RH has advantages of real-time and robustness. The obtained decision plan can fulfill the operational mission in the fewer stages and the “radical-conservative” degree can be adjusted adaptively by parameters.

25 November 2019 in IEEE Access
Verified
Journal Article
Open access
KZKai Zhang
Kai Zhang
DZDeyun Zhou
Deyun Zhou
ZYZhen Yang
Zhen Yang
QPQian Pan
Qian Pan
WKWeiren Kong
Weiren Kong
The weapon target assignment (WTA) problem is the crucial decision support in Command & Control (C2). In the classic WTA model, the point-to-point saturation salvo has a low efficiency-cost ratio when the swarming targets, which have the advantage of low casualty, low cost and recyclable, become the major operational units. The constraint is less studied for the operational intention of the decision-maker. In this paper, a constrained multi-objective weapon target assignment (CMWTA) model is formulated for area targets. The optimization objectives are minimizing collateral damage and resource consumption. The multi-constraint is derived from the actual operational requirements of security evasion, damage threshold, and preference assignment. To solve CMWTA efficiently, a novel multi-objective optimization evolutionary algorithm (MOEA) is proposed to obtain the non-dominated solutions as the alternative plans for the decision-maker. A self-adaptive sorting algorithm is proposed to guarantee the completeness of the Pareto-optimal set, and a cooperative evolutionary mechanism is adopted to strengthen the convergence. For handling multi-constraint, a repair mechanism is proposed to improve the quality of infeasible solutions, and the measurement of constraint violation is designed to evaluate the infeasible solutions. A variant of the convergence metric is introduced to evaluate the algorithms solving multi-objective weapon target assignment (MWTA) problem. The experimental results show the effectiveness and superiority of the proposed approaches.