Cooperative relay transmission is an attractive architecture for underwater acoustic networks. However, designing relay selection policies in the harsh underwater environment is difficult. In this letter, we model relay selection as a contextual bandit problem—an important extension of multi-armed bandit. Through this way, we can achieve relay selection based on a bit of contextual communication environment information about relay nodes instead of instantaneous or statistical channel state information. Our proposed relay selection technique enables highly stable performance of the cooperative system in a complex and changeable underwater environment, and the process of relay selection can be simplified and easily facilitate efficient cooperative transmission. Simulation results illustrate the effectiveness and the robustness of this relay selection technique.

Relay selection solutions for underwater acoustic cooperative networks suffer significant performance degradation as they fail to adapt to incomplete information, noisy interference and overwhelming dynamics. To address this challenge, a hierarchical adversarial multi-armed bandit learning framework by proposing an online reward estimation layer is designed to improve adaptive relay decision control. In online reward estimation layer, adaptive Kalman filter estimator is developed to properly handle noisy observation to support accurate reward. Meanwhile an online predict mechanism is projected for all relays to enrich learning information. Furthermore, based on estimate error variance, an adaptive exploration structure is developed to accelerate the balance between exploration and exploitation. All gathered information are exploited to learn relay quality for the decision-making. Accordingly, we present a Hierarchical Adversarial Bandit Learning (HABL) algorithm to fully exploit the heuristic interaction between the hierarchical framework. HABL integrates reward estimation, information prediction, adaptive exploration and decision making carefully in a holistic algorithm to maximize the learning efficiency. Thereby, the HABL-based relay selection algorithm has higher system throughput and lower communication cost. Further, we rigorously analyze the convergence of HABL algorithm and give its upper bound on the cumulative regret. Finally, extensive simulations elucidate the effectiveness of the HABL.

Pairwise ranging between nodes plays a crucial role in Internet-of-Underwater-Things (IoUT) networks, and it typically impacts the overall performance of such networks. As the sound speed depends on several parameters, time-of-flight-based techniques cannot work well under varying sound speeds in actual underwater conditions. Pairwise ranging algorithms that consider stratification should be studied to improve ranging accuracy. However, there is a tradeoff between underwater acoustic ranging accuracy and number of calculations. This makes it challenging to implement underwater acoustic ranging algorithms in IoUT networks. In this article, we propose an underwater acoustic horizontal ranging algorithm that rapidly and accurately estimates the horizontal range from a sender to a receiver under an arbitrary sound-speed profile in the deep sea. Simulation results show that the proposed algorithm accurately calculates the horizontal range with a low computational complexity. We validate the performance of the proposed algorithm using the data collected during an ultrashort baseline precision test in the South China Sea.

Multi-user cooperative transmission is an attractive architecture for underwater acoustic sensor networks (UASNs). Cooperative transmission depends on careful allocations of resources such as relay selection, but traditional relay selection requires precise measurements of channel state information (CSI), which is infeasible for multi-user cooperative transmission due to the unique features and hardware restrictions of UASNs. In this work, we model multi-user relay selection under a multiuser multi-armed bandit (MU-MAB) framework, whereby users are not provided any prior knowledge about underwater acoustic channel conditions. We first exploit a novel MU-MAB algorithm, DSMU-MAB, for relay selection, assuming that the reward distributions are initially unknown but remain constant. Second, we consider an evolving environment in which the reward distributions undergo changes in time, and DSMU-rMAB, a derivative of DSMU-MAB, is proposed, which can be robust to abrupt changes in underwater communication environments. The proposed algorithms not only help sources find the suitable relays to achieve a high quality transmission and avoid collisions among users but also reduce the mass of information exchanged among users. We established the effectiveness of our proposed algorithms using theoretical and numerical analyses.