An efficient routing protocol is critical for the data transmission of Underwater Wireless Sensor Networks (UWSNs). Aiming to the problem of void region in UWSNs, this paper proposes a reinforcement learning-based opportunistic routing protocol (DROR). By considering the limited energy and underwater environment, DROR is a receiver-based routing protocol, and combines reinforcement learning with opportunistic routing to ensure real-time performance of data transmission as well as energy efficiency. To ensure reliable transmission when encountering void regions, a void recovery mechanism is designed to enable packets to bypass void nodes and continue forwarding. Furthermore, a relative Q-based dynamic scheduling strategy is proposed to ensure that packets can efficiently forward along the global optimal routing path. Simulation results show that the proposed protocol performs well in terms of end-to-end delay, reliability, and energy efficiency in UWSNs.

Underwater wireless sensor networks have emerged as a promising technology for various underwater applications. Considering the characteristics such as limited energy and high end-to-end delay in underwater wireless sensor networks, it is important to design an underwater routing protocol with high energy efficiency, low end-to-end delay and high reliability. Therefore, a Q learning-based routing protocol is proposed in this paper. First, a Q learning-based framework is constructed by considering link connectivity and the information of residual energy, depth and neighboring nodes. The framework enables protocols to adapt to the dynamic environment and facilitate efficient transmission. Furthermore, to address the slow convergence of Q learning in underwater wireless sensor networks, a Q value initialization strategy using layer information is designed to accelerate the convergence speed. In addition, an adaptive discount mechanism and a dynamic learning mechanism are proposed to update Q values for adapting to the changing network topology and improve the reliability of Q values for nodes rarely selected, respectively. Finally, the superior performance of the proposed protocol is evaluated through simulations. Simulation results show that the proposed protocol can still accelerate the convergence speed in reducing the energy tax by 37.16% and 23.08%, and the average end-to-end delay by 29.94% and 16.91% as compared to other Q learning-based routing protocols QELAR and QDAR under dynamic environment, while maintaining a higher packet delivery ratio.

Efficient data transmission plays a crucial role in the applications of underwater wireless sensor networks (UWSNs). In this paper, by considering the differences in transmission requirements for data of varying importance degrees in UWSNs, a multi-agent reinforcement learning-based routing protocol with value of information (MARV) is proposed. First, to distinguish the difference of transmission requirements, we introduce value of information (VoI) to characterize the importance degree of data to reflect the requirement for the real-time characteristic. Moreover, to ensure the efficient routing for different importance degree of data, we establish a multi-agent reinforcement learning-based framework by enabling nodes to learn from the environment and interact with neighbors, and elaborately design a reward function by considering the timeliness and energy efficiency of transmission. In addition, to improve the transmission efficiency, we design a packet holding mechanism by designing a priority list and variable holding interval according to transmission requirements. The simulation results show that the proposed protocol performs well for the transmission of different data.

The channel reservation mechanism is devised to reduce end-to-end delay and improve throughput for underwater acoustic sensor networks (UASNs) with dynamic load. Unfortunately, in dynamic-high-load UASNs, frequent channel reservation will lead to an increase in end-to-end delay and a decrease in throughput due to long propagation delay and collisions. To reduce end-to-end delay and improve throughput, we proposed an adaptive MAC protocol for UASNs with dynamic-high load (DHL-MAC). DHL-MAC performs channel reservation before the data packets are actually generated and avoids collisions to reduce the impact of channel reservation on end-to-end delay and throughput. First, the adaptively allocated time slot for the next time frame is obtained based on the estimation factor and secondary transmission data packets, which could reduce idle waiting time. Then, by merging the channel reservation information for next time frame into the current data frame, channel reservation is performed before data packet generation to reduce waiting delay. Finally, a dynamic time slot schedule scheme is proposed to reduce time frame length and achieve collision avoidance. The performance analysis and simulation results show that our proposed DHL-MAC could efficiently reduce end-to-end delay and improve throughput in dynamic-high-load UASNs. Simulation results show that DHL-MAC can achieve a maximum 95.2% reduction in average end-to-end delay and a maximum 56.2% improvement in throughput upper bound compared with R-MAC.

Many marine scientific and industrial applications in underwater acoustic sensor networks (UASNs) are characterized by dynamic-high load and dense deployment. In such UASNs, the channel reservation mechanism is devised to deal with channel utilization degradation. Unfortunately, channel reservation would lead to an increase in end-to-end delay and a decrease in throughput due to long propagation delay in UASNs. To reduce end-to-end delay and improve throughput, we proposed an efficient distributed MAC protocol for UASNs (ED-MAC). ED-MAC reduces the impact of long propagation delay on channel reservation by removing the channel reservation confirmation process and improving channel reservation efficiency. First, the optimized time slot length for the next time frame is obtained based on data packet number estimate and secondary transmission data packet number, which could reduce idle waiting time. Then, the channel reservation confirmation process is removed by distributed channel reservation mechanism, and efficient channel reservation is achieved through channel reservation packet collision-free transmission and channel multiplexing. Meanwhile, channel reservation is performed before packet generation. Thus, data packet waiting delay is reduced. Finally, a distributed data packet time slot schedule scheme is proposed to achieve collision avoidance between different data packets as well as data packets and channel reservation packets. The performance analysis and simulation results show that our proposed ED-MAC could efficiently reduce end-to-end delay and improve throughput in dynamic-high-load and dense UASNs. Simulation results show that ED-MAC can achieve a maximum 98.7% reduction in average end-to-end delay and a maximum 61.1% improvement in throughput.

Long-term underwater acoustic sensor networks (UASNs) are widely deployed for marine scientific and industrial applications. The widespread ocean currents lead to variations in the communication range of location-fixed nodes equipped with non-omnidirectional transducers, which will reduce the successful packet relaying probability. In order to reduce the impact of ocean current on packet relaying, we propose an adaptive routing protocol for underwater acoustic sensor networks with ocean current (OCAR). By adaptively selecting candidate nodes based on communication range, OCAR improves the successful packet relaying probability in poor communication conditions caused by ocean current. Firstly, the impact of ocean current on communication range is demonstrated by constructing a communication range model and a sea trial. Secondly, successful packet relaying probability is improved by selecting the communicable nodes under ocean current as the candidate nodes. Next, the transmitter-based adaptive routing scheme reduces the number of nodes responding to the relay packet and adaptively reselects the destination node per hop. Therefore, energy consumption is reduced and the packet delivery ratio (PDR) is improved. Finally, routing recovery and candidate set dynamic update are achieved to improve successful packet relaying probability. Simulation results show that, compared with Flood, OCAR can achieve a maximum 98.29% reduction in energy consumption and 37.05% reduction in average end-to-end delay while maintaining close to 100% PDR. In addition, compared with VBF, OCAR can achieve close to 100% PDR when the PDR of VBF is 0, maximum energy consumption reduction of 84.41% and average end-to-end delay reduction of 96.34%.