This paper investigates the problem of anti-jamming communication in dynamic and intelligent jamming environment. A sequential deep reinforcement learning algorithm (SDRLA) without prior information is proposed, and raw spectrum information is used as the input of SDRLA. The proposed SDRLA algorithm mainly contains two parts: Firstly, deep learning and sliding window principle are introduced to identify jamming patterns; Secondly, reinforcement learning is carried out to make on-line channel selection based on recognized jamming patterns. In addition, channel switching cost is introduced for the purpose of formulating the trade-off relationship between throughput and overhead. Taking advantage of both deep learning and reinforcement learning, this method can realize rapid and effective anti-jamming channel selection with no need for modeling the jammer's characteristics. Simulation results show the convergence and effectiveness of the proposed SDRLA algorithm. Compared with single-mode reinforcement learning, our approach can reach better convergence performance and higher utility.

In this letter, the frequency selection problem in jamming environment with large number of optional frequencies is investigated. With numerous optional actions in the wider frequency band scenario, most of existing anti-jamming methods will become ineffective, since the convergence time and computational complexity will grow exponentially with the number of actions. To cope with the above challenge, a novel hierarchical deep reinforcement learning algorithm which does not need to know the jamming patterns and channel model is proposed. The proposed algorithm divides the frequency selection problem in the broadband into two steps via two subnetworks: Firstly, the frequency band is selected by the band selection network, and then the specific frequency is selected in this frequency band by the frequency selection network. Simulation results show that the proposed algorithm avoids multiple different jammings effectively and achieves satisfactory throughput with less calculation.

With the development of access technologies and artificial intelligence, a deep reinforcement learning (DRL) algorithm is proposed into channel accessing and anti-jamming. Assuming the jamming modes are sweeping, comb, dynamic and statistic, the DRL-based method through training can almost perfectly avoid jamming signal and communicate successfully. Instead, in this paper, from the perspective of jammers, we investigate the performance of a DRL-based anti-jamming method. First of all, we design an intelligent jamming method based on reinforcement learning to combat the DRL-based user. Then, we theoretically analyze the condition when the DRL-based anti-jamming algorithm cannot converge, and provide the proof. Finally, in order to investigate the performance of DRL-based method, various scenarios where users with different communicating modes combat jammers with different jamming modes are compared. As the simulation results show, the theoretical analysis is verified, and the proposed RL-based jamming can effectively restrict the performance of DRL-based anti-jamming method.

The primary objective of anti-jamming techniques is to ensure that the transmitted data arrives at the intended receiver without being disturbed or jammed with by any jamming signal or other hostile activities to ensuring the security of the communication system. Deep reinforcement learning (DRL) has been extensively utilized in solving the dynamic spectrum anti-jamming problem. However, most of existing DRL-based algorithms require lots of training time, which fails to adapt the fast-channging jamming environment. Our objective is to find a practical and fast-convergence anti-jamming learning solution. To achieve this, we redesign the DRL algorithm in the following two ways. First, we split the cycle of reinforcement learning into two parts: applying process and training process. Second, we use soft labels instead of rewards which bring more information. We further theoretically show that the information gain can help our proposed algorithm converge faster. Moreover, we also show that our labeled DRL algorithm is better than the idealized DRL-based scheme which can obtain the same information as the soft labels. Simulation results demonstrate that compared with existing DRL-based algorithms, our proposed algorithm reduces the number of iterations by up to 90%.

This paper addresses the challenge of anti-jamming in moving reactive jamming scenarios. The moving reactive jammer initiates high-power tracking jamming upon detecting any transmission activity, and when unable to detect a signal, resorts to indiscriminate jamming. This presents dual imperatives: maintaining hiding to avoid the jammer’s detection and simultaneously evading indiscriminate jamming. Spread spectrum techniques effectively reduce transmitting power to elude detection but fall short in countering indiscriminate jamming. Conversely, changing communication frequencies can help evade indiscriminate jamming but makes the transmission vulnerable to tracking jamming without spread spectrum techniques to remain hidden. Current methodologies struggle with the complexity of simultaneously optimizing these two requirements due to the expansive joint action spaces and the dynamics of moving reactive jammers. To address these challenges, we propose a parallelized deep reinforcement learning (DRL) strategy. The approach includes a parallelized network architecture designed to decompose the action space. A parallel exploration-exploitation selection mechanism replaces the ε-greedy mechanism, accelerating convergence. Simulations demonstrate a nearly 90% increase in normalized throughput.

Pilot-assisted channel estimation techniques are essential for wireless communication systems. Most studies focus on the estimation algorithms and interpolation techniques. However, the design of pilot pattern is often neglected. In this paper, we propose a joint pilot spacing and power optimization scheme based on deep reinforcement learning (DRL) to address the mismatch problem of pilot configuration for nonstationary wireless channel. First, we model the adaptive pilot design decision-making process as a Markov Decision Process (MDP) to reduce pilot overhead and power loss. Then a deep Q-network (DQN) based learning algorithm is proposed to optimize the spacing and power of pilots so as to maximize estimation performance while reducing system cost. Simulation results show that the performance of the proposed approach is better than conventional pilot configuration algorithms. Moreover, we analyze the key factors that affect the performance of the proposed scheme.