Unmanned aerial vehicle (UAV) autonomous tracking and landing is playing an increasingly important role in military and civil applications. In particular, machine learning has been successfully introduced to robotics-related tasks. A novel UAV autonomous tracking and landing approach based on a deep reinforcement learning strategy is presented in this paper, with the aim of dealing with the UAV motion control problem in an unpredictable and harsh environment. Instead of building a prior model and inferring the landing actions based on heuristic rules, a model-free method based on a partially observable Markov decision process (POMDP) is proposed. In the POMDP model, the UAV automatically learns the landing maneuver by an end-to-end neural network, which combines the Deep Deterministic Policy Gradients (DDPG) algorithm and heuristic rules. A Modular Open Robots Simulation Engine (MORSE)-based reinforcement learning framework is designed and validated with a continuous UAV tracking and landing task on a randomly moving platform in high sensor noise and intermittent measurements. The simulation results show that when the moving platform is moving in different trajectories, the average landing success rate of the proposed algorithm is about 10% higher than that of the Proportional-Integral-Derivative (PID) method. As an indirect result, a state-of-the-art deep reinforcement learning-based UAV control method is validated, where the UAV can learn the optimal strategy of a continuously autonomous landing and perform properly in a simulation environment.

The position prediction of marine moving targets based on historical trajectories is an important assistance procedure for marine reconnaissance and surveillance. Limited by satellite access period, space-based historic trajectory data have sparse and uneven intervals. However, most current time-series prediction methods require uniform time intervals. For non-uniform time series data, common processing methods first use the interpolation algorithm to fit historical data, and then carry out predictions based on equal interval data after the uniform sample. The disadvantage is that the accuracy of the interpolation data will limit the prediction accuracy. In addition, the time-series prediction methods represented by the grey model (GM) and autoregressive model (ARM) can only deal with equal-interval time prediction, in which it is hard to satisfy the prediction demand of non-equidistant time. Aiming at the limitations of most time series prediction methods and meeting the requirement of long-term variable duration prediction, a novel trajectory prediction method for sparse and non-uniform time series data based on deep neural networks is proposed. Firstly, to maximize the mining of the original data features, the moving behavior features are extracted from the raw historical track data by calculating the information of position, velocity, and position change for feature extension. Then, because of the temporal coherence of the track data, and inspired by the design idea of local correlation of the convolutional neural network (CNN), the CNN model is used to excavate the navigation rules to achieve position prediction. Finally, training of the network model is accomplished based on historical track samples. The experiments are carried out based on the space-borne automatic identification system (AIS) observation data. Experimental results illustrate that the method behaves better than other methods with the superiority of lower requirements for sampling, stronger adaptability to data characteristics, and higher forecasting accuracy for long-term prediction. When applied to the satellite search of marine moving targets, the track prediction has the potential to reduce the uncertainty of target location and guide satellite searching missions, thereby significantly improving the searching efficiency of targets.

In recent years, with the continuous development of the global economy, maritime transportation has become an essential mode of transportation for both domestic and international trade, making it an urgent task for countries around the world to improve the intelligence level of maritime traffic management. As a key technology in real-time vessel monitoring, traffic hazard early warning, and traffic flow estimation, vessel trajectory prediction has been widely applied across both civil and commercial sectors. To address the problems with existing trajectory prediction methods, such as difficulty in establishing real-time vessel kinematic models, limited understanding of vessel navigation strategies, and poor adaptability for long-term prediction, this paper proposes a deep reinforcement learning-based vessel trajectory prediction method. Firstly, the trajectory prediction problem is formulated as a Markov Decision Process (MDP), thereby transforming the prediction problem into an optimal policy-solving problem of the MDP. Secondly, given the complexity of the trajectory prediction problem, a convolutional neural network (CNN) is adopted to parameterize the policy network, and a CNN-PPO deep reinforcement learning method is used to learn the target vessel’s navigation strategy from historical trajectories. Comparative experimental results show that the proposed algorithm is not only suitable for predicting the target’s position at a specific future moment but also offers clear advantages in trajectory prediction across multiple future moments.