Planning cooperative trajectories in real time is of great importance for multiple unmanned combat aerial vehicles (multi-UCAV) in performing autonomous time-critical cooperative air-to-ground target attack missions. The models of the vehicle, constraints, and a multi-criteria objective function are set up, and then the problem is formulated as a decentralized cooperative receding horizon optimal control problem. An elaborate framework for provable effective decentralized cooperative trajectory planning in two-degree-of-freedom decentralized receding horizon control is presented to provide the robustness and adaptivity for trajectory planning in dynamic and uncertain environments. Furthermore, a two-stage iterative optimization algorithm is developed to solve the problem. The novel algorithm integrates the differential flatness theory to generate trajectories, and the coordination variable-based strategy to coordinate the arrival time of multi-UCAV online, which can improve the computational efficiency and achieve the cooperative objective. The results of numerical simulations show that the proposed approach is feasible and well suited for the real-time implementation in dynamic and uncertain environment.

In addressing the challenge of tracking moving targets at sea, our focus has been directed towards the development of a reconstruction methodology founded upon satellite orbital manoeuvres. This endeavour has led us to devise a predictive model for manoeuvres within a geographic coordinate system, alongside the creation of a three-phase orbital manoeuvre model. A Non-dominant Sorting Adaptive Memetic (NSAM) algorithm is proposed in this paper, which is a two-layer multi-objective optimization algorithm that retains the advantages of evolutionary algorithms based on the population’s evolution and has an excellent local optimization ability of local search algorithms. The proposed algorithm can be used to solve multi-objective optimization problems. By comparing the target observation results before and after the satellite reconstruction simulation, it can be concluded that the orbital manoeuvring can effectively improve the observation probability and observation duration of the target at a certain speed. The orbital manoeuvre model created in this paper provides a certain methodical support for the tracking problem of moving targets at sea.

The Monte Carlo Tree Search (MCTS) has demonstrated excellent performance in solving many planning problems. However, the state space and the branching factors are huge, and the planning horizon is long in many practical applications, especially in the adversarial environment. It is computationally expensive to cover a sufficient number of rewarded states that are far away from the root in the flat non-hierarchical MCTS. Therefore, the flat non-hierarchical MCTS is inefficient for dealing with planning problems with a long planning horizon, huge state space, and branching factors. In this work, we propose a novel hierarchical MCTS-based online planning method named the HMCTS-OP to tackle this issue. The HMCTS-OP integrates the MAXQ-based task hierarchies and the hierarchical MCTS algorithms into the online planning framework. Specifically, the MAXQ-based task hierarchies reduce the search space and guide the search process. Therefore, the computational complexity is significantly reduced. Moreover, the reduction in the computational complexity enables the MCTS to perform a deeper search to find better action in a limited time. We evaluate the performance of the HMCTS-OP in the domain of online planning in the asymmetric adversarial environment. The experiment results show that the HMCTS-OP outperforms other online planning methods in this domain.

Unmanned ground vehicles (UGVs) have been widely used in security patrol. The existence of two potential opponents, the malicious teammate (cooperative) and the hostile observer (adversarial), highlights the importance of privacy-preserving planning under contested environments. In a cooperative setting, the disclosure of private information can be restricted to the malicious teammates. In adversarial setting, obfuscation can be added to control the observability of the adversarial observer. In this paper, we attempt to generate opponent-aware privacy-preserving plans, mainly focusing on two questions: what is opponent-aware privacy-preserving planning, and, how can we generate opponent-aware privacy-preserving plans? We first define the opponent-aware privacy-preserving planning problem, where the generated plans preserve admissible privacy. Then, we demonstrate how to generate opponent-aware privacy-preserving plans. The search-based planning algorithms were restricted to public information shared among the cooperators. The observation of the adversarial observer could be purposefully controlled by exploiting decoy goals and diverse paths. Finally, we model the security patrol problem, where the UGV restricts information sharing and attempts to obfuscate the goal. The simulation experiments with privacy leakage analysis and an indoor robot demonstration show the applicability of our proposed approaches.

Hierarchical skill learning is an important research direction in human intelligence. However, many real-world problems have sparse rewards and a long time horizon, which typically pose challenges in hierarchical skill learning and lead to the poor performance of naive exploration. In this work, we propose an algorithmic framework called surprise-based hierarchical exploration for model and skill learning (Surprise-HEL). The framework leverages the surprise-based intrinsic motivation for improving the efficiency of sampling and driving exploration. It also combines the surprise-based intrinsic motivation and the hierarchical exploration to speed up the model learning and skill learning. Moreover, the framework incorporates the reward independent incremental learning rules and the technique of alternating model learning and policy update to handle the changing intrinsic rewards and the changing models. These works enable the framework to implement the incremental and developmental learning of models and hierarchical skills. We tested Surprise-HEL on a common benchmark domain: Household Robot Pickup and Place. The evaluation results show that the Surprise-HEL framework can significantly improve the agent’s efficiency in model and skill learning in a typical complex domain.