Underwater sensor networks (UWSNs) are vulnerable to jamming attacks due to the narrow frequency bandwidth and the fast fading channels. In this paper, we propose a reinforcement learning (RL)-based antijamming relay scheme for UWSNs that enables an underwater relay to decide whether to leave the heavily jammed location and choose the relay power based on the state that consists of the bit error rate of the previous transmission, the previous relay power, the current transmit power of the sensor, and the jamming power measured by the relay node. We also propose a deep-RL-based relay scheme to further improve the relay performance for the node that supports deep learning computation. We discuss the computational complexity of the deep-RL-based relay scheme and provide the relay performance bound regarding the bit error rate, energy consumption, and utility of the relay node. Experiments taken in a nonanechoic pool with underwater transducers against smart jamming attacks verify the analysis results. According to the experimental results, the proposed relay scheme can improve the relay performance compared with the benchmark underwater relay schemes.

In this paper, we propose a reinforcement learning-based adaptive modulation and coding scheme for underwater communications. More specifically, based on the network states such as the quality of service requirement of the sensing message, the previous transmission quality and the energy consumption. This scheme applies reinforcement learning to choose the modulation and coding policy in a dynamic underwater communication system. We provide the performance bound of this scheme and perform experiments in both pool and sea environments. The experimental data were collected and post-processed. Compared with the benchmark schemes, this scheme can improve the throughputs and reduce the BER with less energy consumption.

In this paper, we propose a physical (PHY)-layer authentication framework to detect spoofing attacks in underwater sensor networks. This scheme exploits the power delay profile of the underwater acoustic channel to discriminate the sensors and applies reinforcement learning (RL) to choose the authentication parameter without being aware of the network and spoofing model. We propose an RL based authentication scheme to provide light-weight spoofing detection and a deep RL based authentication scheme to further improve the authentication accuracy for sinks that support deep learning. Experiment results show that this scheme improves the spoofing detection accuracy and increases the utility of the network compared with the benchmark PHY-layer authentication.

In mobile edge computing (MEC) systems, multiple unmanned aerial vehicles (UAVs) can be utilized as aerial servers to provide computing, communication, and storage services for edge users, called UAV-assisted MEC, which has emerged as a promising technology to improve both the computing and communication performances. Unlike existing works without considering jamming attacks, we investigate a multi-UAV-assisted-MEC scenario under multiple malicious jammers and then propose a resource management approach with the objective of minimizing both the system energy consumption and latency. Due to the time-varying nature of communication environments, we design a multi-agent deep reinforcement learning (MADRL)-based resource management approach to dynamically adjust the CPU frequency, communication bandwidth, and channel access selection of UAVs to enhance the system performance against jamming attacks. On this basis, in order to enhance the algorithm learning efficiency, we propose a multi-agent twin-delayed deep deterministic policy algorithm in combination with the prioritized experience replay mechanism (PER-MATD3) to effectively search for the joint resource management strategy under high-dimensional state and action spaces, where the time-varying channel state information and imperfect attack behavior information are also effectively trained to improve the learning capacity and convergence speed. Simulation and experimental results verify that the proposed approach can significantly decrease the overall system latency (i.e., computing and communication latency) and energy consumption compared to other benchmark algorithms under different real-world settings.

Litopenaeus vannamei, as the main target of shrimp farming in China, has a great market prospect and high economic value. However, domestic Litopenaeus vannamei farming and feeding efficiency are low, mainly through the use of hand tools by farmers to complete the feed throwing work, and the feed cost is high. Therefore, in view of the high density and low visibility of culture ponds in Litopenaeus vannamei aquaculture industry, it is impossible to judge the quantity and time of feed feeding by visual way. In this study, based on the analysis of acoustic feeding signal characteristics of Litopenaeus vannamei, accurate and intelligent control of feed amount is proposed, so as to reduce the breeding cost and obtain environmental benefits. This study designed a system for collecting and analyzing the acoustic signals of the feeding motion of Litopenaeus vannamei. Based on passive acoustic technology, the motion acoustic signal of Litopenaeus vannamei is extracted when starving. Through time domain and frequency domain research, the characteristic parameters of the motion acoustic signal of the specific body type of Litopenaeus vannamei are determined. The signal duration range is 8-10ms, and the average range of the resonance band is 2-10KHz. Furthermore, the dual-threshold method is further used to detect the effective range of the feeding signals, and then the number of acoustic signals is obtained, and the corresponding magnitude relationship between the characteristic parameters and the starvation degree of Litopenaeus vannamei is established, which can be used as a basis for judging intelligent feeding on demand. The research provides theoretical support for the feasibility of studying the feeding patterns of Litopenaeus vannamei through underwater passive acoustic detection.

In this letter, we proposed an efficient underwater acoustic (UWA) image communication algorithm based on reinforcement learning which can improve the image quality while reduce the energy consumption and time delay in fast time variant UWA channels. In the proposed algorithm, the received image quality and other communication performance parameters are estimated at the sink continuously and then feedback to the sensor by an independent channel in order to avoid bandwidth loss caused by large time delay. At the sensor, the most suitable modulation and coding method is chosen to maximize a special designed value function in order to achieve the best efficient underwater image communication. Sea test results show that the proposed UWA image communication algorithm can reduce the bit-error rate by 3.1 dB, the energy consumption of the sensor by 26.9% and the time delay by 58.2%. The proposed algorithm can also shorten the convergence time by 47.4% compared with the model-free reinforcement learning underwater communication algorithm.