Invalid action masking is a practical technique in deep reinforcement learning to prevent agents from taking invalid actions. Existing approaches rely on action masking during policy training and utilization. This study focuses on developing reinforcement learning algorithms that incorporate action masking during training but can be used without action masking during policy execution. The study begins by conducting a theoretical analysis to elucidate the distinction between naive policy gradient and invalid action policy gradient. Based on this analysis, we demonstrate that the naive policy gradient is a valid gradient and is equivalent to the proposed composite objective algorithm, which optimizes both the masked policy and the original policy in parallel. Moreover, we propose an off-policy algorithm for invalid action masking that employs the masked policy for sampling while optimizing the original policy. To compare the effectiveness of these algorithms, experiments are conducted using a simplified real-time strategy (RTS) game simulator called Gym-$\mu$RTS. Based on empirical findings, we recommend utilizing the off-policy algorithm for addressing most tasks while employing the composite objective algorithm for handling more complex tasks.

Unmanned aerial vehicles (UAVs) have numerous applications in reconnaissance, search, and rescue missions. However, searching underwater areas remains a challenge for UAVs. For anti-submarine search tasks, this paper proposes using a mixed swarm of UAVs and unmanned surface vehicles (USVs). To generate vehicle paths with the goal of minimizing the travel time while achieving complete coverage, a two-phase strategy is developed. In the first phase, optimal waypoints are determined to balance the task volume among the UAVs and USVs. A mapping model is constructed to represent the relationships between the different unmanned platforms and the mission environment. In the second phase, the search trajectory for each UAV/USV is generated by combining waypoint clustering and closed-loop trajectory generation. A detailed simulation study confirms the feasibility and flexibility of the proposed path planning method when applied to cross-domain swarms of different types and sizes for anti-submarine missions. In addition, the simulation results show that cross-domain cooperative planning for unmanned platforms may have advantages over single-domain planning in some scenarios.

This paper develops an autonomous cooperative interception method with angle constraints, allowing a swarm of unmanned aerial vehicles (UAVs) to simultaneously intercept an incoming maneuvering target from predesigned encirclement angles. To this end, a decision-making framework based on the State-Event-Condition-Action (SECA) model is proposed to drive the UAV's autonomous behavior, and a series of SECA rules are defined to support the state transition. Additionally, a waypoint-following control method is derived for the UAV swarm to satisfy the encirclement angle constraints. The encirclement angles and desired waypoints are well-designed to implement interception tactics in different states. The coordinated interception time does not need to be predetermined and is replaced by defined distance variables under the virtual leader-based scheme. Simulation results demonstrate that the UAV swarm tracking the desired waypoint can satisfy the angle constraints for intercepting the incoming maneuvering target.

Source localization based on the hybrid time-difference-of-arrival (TDOA) and frequency-difference-of-arrival (FDOA) measurements from distributed sensors is an essential problem in wireless sensor networks (WSNs). In this paper, we mainly study the optimal sensor deployment and velocity configuration of UAV swarms mounted with TDOA and FDOA based sensors. Explicit solutions of optimal sensor deployment and velocity configuration are acquired in both static and movable source scenarios based on the Fisher information matrix (FIM). Both centralized and decentralized localization are explored to meet different types of localization methods. Path planning problem of UAV swarms in TDOA/FDOA localization is also studied with constraints. Simulations verify its efficiency with path planning in TDOA and FDOA localization.

Given that signal is weakened to a certain extent in the process of noise suppression using mainstream method, and that new noise is introduced by signal processing system, causing the decrease of detection performance, to improve the performance of detection to BPSK signal under the condition of strong noise and no prior information, the detection algorithm of BPSK signal of bistable stochastic resonance model based on scale change is proposed in this study. Using classical bistable stochastic resonance (BSR) system, only low-amplitude and low-frequency periodic signal can be processed. Scale change is first made to BSR in this study, verifying that BSR can be applied to high-frequency BPSK signal under high sampling frequency condition, and nonlinear threshold detection system is designed following Neyman-Pearson criterion to deduce and quantitatively show error rate of detector. Besides, complete flow for signal detection was built by taking it as feedback quantity to adjust the system parameters adaptively. Scale change feasibility and applicability of algorithm proposed in this study were verified through simulation experiment, which lays the theoretical basis for the detection of weak BPSK signal under low signal-to-noise ratio (SNR).