In this article, we show the development and implementation of the Internet of Maritime Things (IoMT) platform that supports long-range and high-rate communication for remote and online marine water quality monitoring. We first develop the IoMT device and gateway that interwork with international standard Internet-of-Things (IoT) software platforms for universal utilization in the field of marine monitoring. The proposed IoMT device and gateway play a role similar to the IoT device and gateway on land, but it is optimized for marine use with consideration of the practical characteristics of using sensors in the ocean. Furthermore, to realize long-range and high-rate communication between marine sensors and a control center (CC) on land, an automatic beam adjustment system for directional antennas is developed. This enables the use of directional antennas, which can support long-range and high-rate communication by automatically adjusting the beam angle between the antennas while exchanging their control information, despite the antennas being shifted by undulation from the sea. Finally, we conduct real sea experiments to identify the performances of the proposed IoMT platform in various environments. The results show that the proposed IoMT platform can collect data from marine sensors as well as high definition camera video from distances of 25 and 40 km to the CC on land under one-hop and two-hop transmissions, respectively. These results indicate that the proposed IoMT platform provides at least twofold performance improvement in terms of communication coverage under a similar data rate condition, compared with existing commercial wireless networks (e.g., LTE in the ocean) not utilizing satellite communications.

Integrating underwater optical wireless communication (UOWC) with marine applications such as underwater sensors, buoys, and marine surface vehicles (MSVs), requires the aligning and maintaining of the optical beam between the transmitter and receiver for point-to-point (P2P) UOWC during the data transmission. An additional issue is the difficulty in exchanging batteries for marine applications because of the relatively high costs and risks compared with battery exchanging in terrestrial applications. This study seeks to resolve these issues via joint optimization of the beam divergence angle and transmission power level in an underwater sensor (i.e., transmit node) to maintain a seamless connection with an MSV (i.e., receive node) while minimizing the battery consumption of the sensor. In this regard, we adopt a hybrid underwater acoustic-optical communication system, where acoustic and optical communications are used for low-rate control data transmission and high-rate sensing data transmission, respectively. Under this framework, we propose a two-phase deep reinforcement learning (TPDRL) algorithm considering two agents (inner and outer) that determine different actions using an underwater sensor. Specifically, the primary role of the outer agent is to choose a transmission power level based on the long-term signal-to-noise ratio (SNR) between the underwater sensor and MSV. Next, the inner agent finds the beam divergence angle for the given transmission power (selected from the outer agent) based on the short-term instantaneous SNR. Simulation results demonstrate that the proposed TPDRL algorithm enables seamless and energy-efficient P2P UOWC, performing better than the algorithm with only the inner agent and other existing algorithms.

In this work, we study learning-aided adaptive control of optical beam alignment to maintain a seamless connection with high communication performance in a point-to-point (P2P) underwater optical wireless communication (UOWC). To this end, we propose a two-step two-agent deep reinforcement learning (TSTA-DRL) algorithm that enables an underwater sensor (US) installed on the seabed to sequentially determine the beam orientation (BO) and beam divergence (BD) angles for transmitting its sensing data to an unmanned surface vehicle (USV) that may irregularly shake above the sea level. Specifically, the proposed TSTA-DRL algorithm includes two DRL agents: BO and BD. The BO agent selects the BO angle to point the optical beam of the US toward the USV to perform beam alignment between the US and USV. Moreover, given the BO angle determined by the BO agent, the BD agent chooses the BD angle to maximize the signal-to-noise ratio (SNR) while maintaining the seamless optical link between the two nodes. For the practical application of the proposed algorithm, movement data of the USV measured in the South Sea of Korea are utilized for training the proposed algorithm. The simulation results demonstrate that the proposed TSTA-DRL algorithm achieves the highest SNR while maintaining a stable UOWC link compared with the existing algorithms.

In this work, we investigate a cost-efficient strategy to implement robust online overhead transmission line monitoring for stable energy transfer in smart power grid. As a first step, we first present an analytical framework to evaluate communication performances of wireless sensor networks installed for the real-time monitoring of overhead power transmission line, wherein each tower performs fountain code (FC)-based transmission for guaranteeing transmission reliability between each tower and a substation. Then, we evaluate communication performances given in terms of successful FC-based transmission probability and its end-to-end delay for each tower. Further, our analysis helps formulate an optimization problem that aims at minimizing the total number of activated wide area network (WAN) communication modules deployed on towers(i.e., minimizing the total monetary cost required to use WAN communications), while simultaneously supporting three different types of communication quality of service (QoS) requirements (e.g., reliability, delay, robustness).To find the solution of the problem, we propose the Path Decision in the order of the Farthest Towers (PDFP) algorithm, which enables to find a suboptimal-solution of the problem in real-time. The key idea of PDFT algorithm is that paths (i.e., whether to use a WAN communication module) are sequentially determined in the order of towers farthest from the substation. To identify the validity of the proposed algorithm, we conduct performance evaluations with using public topological and geometrical information of an actual national power grid in Korea. Performance evaluation results show that the proposed PDFT algorithm can find towers that need to activate their WAN communication modules for guaranteeing diverse QoS requirements in real-time, as compared to an optimal-solution algorithm without significant difference in total number of activated WAN communication modules.

In this study, the problem of dynamic channel access in distributed underwater acoustic sensor networks (UASNs) is considered. First, we formulate the dynamic channel access problem in UASNs as a multi-agent Markov decision process, wherein each underwater sensor is considered an agent whose objective is to maximize the total network throughput without coordinating with or exchanging messages among different underwater sensors. We then propose a distributed deep Q-learning-based algorithm that enables each underwater sensor to learn not only the behaviors (i.e., actions) of other sensors, but also the physical features (e.g., channel error probability) of its available acoustic channels, in order to maximize the network throughput. We conduct extensive numerical evaluations and verify that the performance of the proposed algorithm is similar to or even better than the performance of baseline algorithms, even when implemented in a distributed manner.

In this work, we consider a point-to-point underwater optical wireless communication scenario where an underwater sensor (US) transmits its sensing data to a remotely operated vehicle (ROV). Before the US transmits its data to the ROV, the ROV performs simultaneous lightwave information and power transfer (SLIPT), delivering both control data and lightwave power to the US. Under the considered scenario, our objective is to maximize energy harvesting at the US while supporting predetermined communication performance between the two nodes. To achieve this objective, we develop a hierarchical deep Q-network (DQN)–deep deterministic policy gradient (DDPG)-based online algorithm. This algorithm involves two reinforcement learning agents: the ROV and US. The role of the ROV agent is to determine an optimal beam-divergence angle that maximizes the received optical signal power at the US while ensuring a seamless optical link. Meanwhile, the US agent, which is influenced by the decision of the ROV agent, is responsible for determining the time-switching and power-splitting ratios to maximize energy harvesting without compromising the required communication performance. Unlike existing studies that do not account for adaptive parameter control in underwater SLIPT, the proposed algorithm’s adaptive nature allows for the dynamic fine-tuning of optimization parameters in response to varying underwater environmental conditions and diverse user requirements.