In this study, we introduce a novel method for an online trajectory replanning approach for fixed-wing Unmanned Aerial Vehicles (UAVs). Our method integrates moving obstacle predictions within a gradient-based optimization framework. The trajectory is represented by uniformly discretized waypoints, which serve as the optimization variables within the cost function. This cost function incorporates multiple objectives, including obstacle avoidance, kinematic and dynamic feasibility, similarity to the reference trajectory, and trajectory smoothness. To enhance prediction accuracy, we combine physics-based and pattern-based methods for predicting obstacle movements. These predicted movements are then integrated into the online trajectory replanning framework, significantly enhancing the system’s safety. Our approach provides a robust solution for navigating dynamic environments, ensuring both optimal and secure UAV operation.

In the field of real-time autonomous decision-making for Unmanned Combat Aerial Vehicles (UCAVs), reinforcement learning is widely used to enhance their decision-making capabilities in high-dimensional spaces. These enhanced capabilities allow UCAVs to better respond to the maneuvers of various opponents, with the win rate often serving as the primary optimization metric. However, relying solely on the terminal outcome of victory or defeat as the optimization target, but without incorporating additional rewards throughout the process, poses significant challenges for reinforcement learning due to the sparse reward structure inherent in these scenarios. While algorithms enhanced with densely distributed artificial rewards show potential, they risk deviating from the primary objectives. To address these challenges, we introduce a novel approach: the homotopy-based soft actor–critic (HSAC) method. This technique gradually transitions from auxiliary tasks enriched with artificial rewards to the main task characterized by sparse rewards through homotopic paths. We demonstrate the consistent convergence of the HSAC method and its effectiveness through deployment in two distinct scenarios within a 3D air combat game simulation: attacking horizontally flying UCAVs and a combat scenario involving two UCAVs. Our experimental results reveal that HSAC significantly outperforms traditional algorithms, which rely solely on using sparse rewards or those supplemented with artificially aided rewards.