Cooperative relay transmission is an attractive architecture for underwater acoustic networks. However, designing relay selection policies in the harsh underwater environment is difficult. In this letter, we model relay selection as a contextual bandit problem—an important extension of multi-armed bandit. Through this way, we can achieve relay selection based on a bit of contextual communication environment information about relay nodes instead of instantaneous or statistical channel state information. Our proposed relay selection technique enables highly stable performance of the cooperative system in a complex and changeable underwater environment, and the process of relay selection can be simplified and easily facilitate efficient cooperative transmission. Simulation results illustrate the effectiveness and the robustness of this relay selection technique.

This article deals with the task assignment problem for multiple autonomous underwater vehicles to efficiently collect underwater data from sensors. We formulate a unified framework to consistently address the heterogeneous task assignment problem (nonemergency and emergency cases) without strictly distinguishing the mixed cases. First, a unified problem, which bridges the gap between different constraints and optimization objectives of different cases, is constructed. Then, the proposed reinforced self-organizing mapping algorithm is reinforced in three aspects: the regional learning rate, the self-configuring neuron (SCN) strategy, and the workload balance mechanism. Specifically, the proposed regional learning rate comprehensively considers the individual worth of tasks and the topology to generate the regional learning rate of dynamic task regions, which consists of dynamic remaining tasks and the reconstructed topology. Based on this idea, the constructed unified problem can be solved consistently. Furthermore, the proposed SCN strategy optimizes the neuron population both in quality and quantity, and guides the update of neurons with enriched historical information to improve the mapping ability. This strategy greatly improves learning efficiency and applicability in a wide range of scenarios. Meanwhile, the proposed workload balance mechanism takes into consideration of both the work capability and consumed energy to extend the continuous working capability. The numerical results validate the effectiveness and adaptability of the proposed unified task assignment framework.

Relay selection solutions for underwater acoustic cooperative networks suffer significant performance degradation as they fail to adapt to incomplete information, noisy interference and overwhelming dynamics. To address this challenge, a hierarchical adversarial multi-armed bandit learning framework by proposing an online reward estimation layer is designed to improve adaptive relay decision control. In online reward estimation layer, adaptive Kalman filter estimator is developed to properly handle noisy observation to support accurate reward. Meanwhile an online predict mechanism is projected for all relays to enrich learning information. Furthermore, based on estimate error variance, an adaptive exploration structure is developed to accelerate the balance between exploration and exploitation. All gathered information are exploited to learn relay quality for the decision-making. Accordingly, we present a Hierarchical Adversarial Bandit Learning (HABL) algorithm to fully exploit the heuristic interaction between the hierarchical framework. HABL integrates reward estimation, information prediction, adaptive exploration and decision making carefully in a holistic algorithm to maximize the learning efficiency. Thereby, the HABL-based relay selection algorithm has higher system throughput and lower communication cost. Further, we rigorously analyze the convergence of HABL algorithm and give its upper bound on the cumulative regret. Finally, extensive simulations elucidate the effectiveness of the HABL.

This study deals with the task assignment problem of heterogeneous unmanned aerial vehicle (UAV) system with the limited resources and task priority constraints. The optimization model which comprehensively considers the resource consumption, task completion effect, and workload balance is formulated. Then, a concept of fuzzy elite degree is proposed to optimize and balance the transmission of good genes and the variation strength of population during the operations of algorithm. Based on the concept, we propose the fuzzy elite strategy genetic algorithm (FESGA) to efficiently solve the complex task assignment problem. In the proposed algorithm, two unlock methods are presented to solve the deadlock problem in the random optimization process; a sudden threat countermeasure (STC) mechanism is presented to help the algorithm quickly respond to the change of task environment caused by sudden threats. The simulation results demonstrate the superiority of the proposed algorithm. Meanwhile, the effectiveness and feasibility of the algorithm in workload balance and task priority constraints are verified.