The stealth aircraft, studied in this article, plans a low observability trajectory to evade radars tracking, considering probability of detection and system constraints. An elaborate framework of planning low observability trajectory, which integrated the models of the stealth aircraft and radars, the theory of multi-phase optimal control and the algorithms of adaptive pseudospectral method, is presented in this article. The constraints and temporal features of low observability trajectory are modeled. The optimal objectives of the flight time, the total fuel consumption, and stealth are defined and synthesized. The trajectory planning problem then was formulated as a multi-objective multi-phase optimal control problem, which can availably grasp the radar tracking features to ensure safety in the whole flight process. A hybrid heuristic adaptive pseudospectral method is developed to solve the trajectory planning problem. The novel algorithm integrates prior knowledge, error estimate and solution regularity for adaptive strategy, which improves computational efficiency and convergence speed. The results of experiments show that the proposed method is feasible and the radar tracking features are effectively utilized to optimize the comprehensive efficiency of penetration.

Multiagent systems (MASs) have received extensive attention in a variety of domains, such as robotics and distributed control. This paper focuses on how independent learners (ILs, structures used in decentralized reinforcement learning) decide on their individual behaviors to achieve coherent joint behavior. To date, Reinforcement learning(RL) approaches for ILs have not guaranteed convergence to the optimal joint policy in scenarios in which communication is difficult. Especially in a decentralized algorithm, the proportion of credit for a single agent’s action in a multiagent system is not distinguished, which can lead to miscoordination of joint actions. Therefore, it is highly significant to study the mechanisms of coordination between agents in MASs. Most previous coordination mechanisms have been carried out by modeling the communication mechanism and other agent policies. These methods are applicable only to a particular system, so such algorithms do not offer generalizability, especially when there are dozens or more agents. Therefore, this paper mainly focuses on the MAS contains more than a dozen agents. By combining the method of parallel computation, the experimental environment is closer to the application scene. By studying the paradigm of centralized training and decentralized execution(CTDE), a multi-agent reinforcement learning algorithm for implicit coordination based on TD error is proposed. The new algorithm can dynamically adjust the learning rate by deeply analyzing the dissonance problem in the matrix game and combining it with a multiagent environment. By adjusting the dynamic learning rate between agents, coordination of the agents’ strategies can be achieved. Experimental results show that the proposed algorithm can effectively improve the coordination ability of a MAS. Moreover, the variance of the training results is more stable than that of the hysteretic Q learning(HQL) algorithm. Hence, the problem of miscoordination in a MAS can be avoided to some extent without additional communication. Our work provides a new way to solve the miscoordination problem for reinforcement learning algorithms in the scale of dozens or more number of agents. As a new IL structure algorithm, our results should be extended and further studied.

Modern war is highly dependent on intelligent, unmanned combat systems. Since many intelligent, unmanned combat systems have network attributes, it is meaningful to research combat systems from the perspective of complex network. Heterogeneous network provides a suitable model to describe real combat network. Previous studies of combat network only concentrate on homogeneous networks. However, on the real battlefield, military networks are composed of a large number of heterogeneous nodes and edges with different functions. In the paper, a superior, intelligent, heterogeneous combat network disintegration strategy (HDGED) are obtained by DQN, which embeds heterogeneous networks into a low-dimensional representation vector as input, rather than ignore the differences of the nodes and their connections. A method of heterogeneous graph embedding is first introduced, which adopts type encoding and aggregation. Besides, a normalized combat capability index was designed, which could assess the performance of the dynamic heterogeneous combat networks. On this basis, HDGED was experimented on networks with uneven node combat capabilities and the results show that HDGED has improved disintegration effectiveness for heterogeneous networks of different sizes compared with traditional methods. Our work provides a new approach to realize the disintegration of heterogeneous combat networks by deep reinforcement learning, which is of great significance for optimizing the command operation process, and deserves further study.

In this paper, more efficient allocation of forces is analyzed in the future air confrontation among unmanned aerial vehicle swarms. A novel method is proposed for swarm confrontation based on the Lanchester law and Nash equilibrium. Due to the huge number of unmanned aerial vehicles, it is not beneficial to deploy UAV forces in swarm confrontation. Moreover, unmanned aerial vehicles do not have high maneuverability in collaboration. Therefore, we propose to divide the swarms of unmanned aerial vehicles into groups, so that swarms of both sides can fight in different battlefields, which could be considered as a Colonel Blotto Game. Inspired by the double oracle algorithm, a Nash equilibrium solving method is proposed to searched for the best force allocation of the swarm confrontation. In addition, this paper proposes the concept of the boundary contact rate and carries out quantitative numerical analysis with the Lanchester law. Experiments reveal the relationship between the boundary contact rate and the optimal strategy of swarm confrontation, which could guide the force allocation in future swarm confrontation. Furthermore, the effectiveness of the division method and the double oracle-based equilibrium solving algorithm proposed in this paper is verified.

Current graph-embedding methods mainly focus on static homogeneous graphs, where the entity type is the same and the topology is fixed. However, in real networks, such as academic networks and shopping networks, there are typically various types of nodes and temporal interactions. The dynamical and heterogeneous components of graphs in general contain abundant information. Currently, most studies on dynamic graphs do not sufficiently consider the heterogeneity of the network in question, and hence the semantic information of the interactions between heterogeneous nodes is missing in the graph embeddings. On the other hand, the overall size of the network tends to accumulate over time, and its growth rate can reflect the ability of the entire network to generate interactions of heterogeneous nodes; therefore, we developed a graph dynamics model to model the evolution of graph dynamics. Moreover, the temporal properties of nodes regularly affect the generation of temporal interaction events with which they are connected. Thus, we developed a node dynamics model to model the evolution of node connectivity. In this paper, we propose DHGEEP, a dynamic heterogeneous graph-embedding method based on the Hawkes process, to predict the evolution of dynamic heterogeneous networks. The model considers the generation of temporal events as an effect of historical events, introduces the Hawkes process to simulate this evolution, and then captures semantic and structural information based on the meta-paths of temporal heterogeneous nodes. Finally, the graph-level dynamics of the network and the node-level dynamics of each node are integrated into the DHGEEP framework. The embeddings of the nodes are automatically obtained by minimizing the value of the loss function. Experiments were conducted on three downstream tasks, static link prediction, temporal event prediction for homogeneous nodes, and temporal event prediction for heterogeneous nodes, on three datasets. Experimental results show that DHGEEP achieves excellent performance in these tasks. In the most significant task, temporal event prediction of heterogeneous nodes, the values of precision@2 and recall@2 can reach 30.23% and 10.48% on the AMiner dataset, and reach 4.56% and 1.61% on the DBLP dataset, so that our method is more accurate at predicting future temporal events than the baseline.

This study addressed a problem of rapid velocity consensus within a swarm of unmanned aerial vehicles. Our analytical framework was based on tools using matrix theory and algebraic graph theory. We established connections between algebraic connectivity and the speed of converging on a velocity. The relationship between algebraic connectivity and communication cost was established. To deal with the trade-off among algebraic connectivity, convergence speed and communication cost, we propose a distributed small world network construction method. The small world network characteristics expedite the convergence speed toward consensus in the unmanned aerial vehicle swarm. Eventually, our method greatly sped up the consensus velocities in the unmanned aerial vehicle swarms at a lower communication cost than other methods required.