With the development of information technology, the degree of intelligence in air confrontation is increasing, and the demand for automated intelligent decision-making systems is becoming more intense. Based on the characteristics of over-the-horizon air confrontation, this paper constructs a super-horizon air confrontation training environment, which includes aircraft model modeling, air confrontation scene design, enemy aircraft strategy design, and reward and punishment signal design. In order to improve the efficiency of the reinforcement learning algorithm for the exploration of strategy space, this paper proposes a heuristic Q-Network method that integrates expert experience, and uses expert experience as a heuristic signal to guide the search process. At the same time, heuristic exploration and random exploration are combined. Aiming at the over-the-horizon air confrontation maneuver decision problem, the heuristic Q-Network method is adopted to train the neural network model in the over-the-horizon air confrontation training environment. Through continuous interaction with the environment, self-learning of the air confrontation maneuver strategy is realized. The efficiency of the heuristic Q-Network method and effectiveness of the air confrontation maneuver strategy are verified by simulation experiments.
1 February 2021 in IOP Conference Series: Earth and Environmental Science
Employing a temporary array under a high-speed railway viaduct deployed by Peking University, we observe how the spectral characteristics vary with changes of the speed and the model of the train as well as the rail and groundsill by using the clustering algorithm K-Means. For a train in uniform motion, the spectrum of high-speed rail seismic wave is mainly composed of nearly equally spaced peaks and its fundamental frequency is equal to the ratio of the train speed to the carriage length. We reduce the influence from the train speed by aligning the fundamental frequency to make the spectrum pattern clear and easy for comparison. Clustering results show that the spectra of the high-speed rail seismic events have stable patterns under the same train model, rail and groundsill conditions; the stable spectrum patterns change significantly with the changes of the train model, rail and groundsill conditions; monitoring the stable spectral characteristics might be used in safety control of high-speed rail. We apply the clustering method on all the stations of the array. In order to obtain spectra with higher signal to noise ratio at farther stations, we consider the variation of train type and stack the spectrums of three components of high-speed rail seismic signal produced by trains of the same type on the same station. Using the clustering algorithm, we get the regular pattern of how the three component spectra vary with the train type and station position. Based on the above research on the characteristics of high-speed rail seismic spectra and their variation, we propose the concept of 4D ground-frequency map, and discuss its practicability in monitoring the status of high-speed rail and its surrounding media.