Underwater acoustic (UWA) adaptive modulation (AM) requires feedback about channel state information (CSI) but the long propagation delays and time-varying features of UWA channels can cause the CSI feedback to be outdated. When the AM mode is selected by outdated CSI, the mismatch between the outdated CSI and the actual CSI during transmission degrades the performance and can even lead to communication failure. Reinforcement learning has the ability to learn the relationships between adaptive systems and the environment. This paper proposes a deep Q-network (DQN)-based AM method for UWA communication that uses a series of outdated CSI as the system input. Our study showed that it could extract channel information and select appropriate modulation modes in the expected channels more effectively than single Q-learning (QL) without needing a deep neural network structure. Furthermore, to mitigate any decision bias that was caused by partial observations of UWA channels, we improved the DQN-based AM by integrating a long short-term memory (LSTM) neural network, named LSTM-DQN-AM. The proposed scheme could enhance the DQN’s ability to remember and process historical input channel information, thus strengthening its relationship mapping ability for state-action pairs and rewards. The pool and sea experimental results demonstrated that the proposed LSTM-DQN-AM outperformed DQN-, QL- and threshold-based AM methods.

Plug-in Hybrid Electric Vehicles (PHEVs), as a new type of environmental-friendly low cost transportation, have attracted growing interests for logistics. The path-planning optimization for PHEV has become a major challenge. In fact, PHEV-based routing optimization is a type of hybrid vehicle routing problem (HVRP). Compared with the traditional Traveling Salesman Problem (TSP) and Vehicle Routing Problem (VRP), the PHEV routing problem should consider more constraints, such as time limits, capacity constraints (including fuel tank capacity and battery capacity), electric stations, fuel stations and so forth. In this paper, a Mixed Integer Linear Programming formulation is presented and a novel hybrid metaheuristic approach (HMA_SVND) is proposed. Our method is a combination of memetic algorithm (MA), sequential variable neighborhood descent (SVND) and a revised 2_opt method. Comparative studies show that our proposed method outperformed previous works.

In underwater sensor networks (UWSNs), the unique characteristics of acoustic channels have posed great challenges for the design of medium access control (MAC) protocols. The long propagation delay problem has been widely explored in recent literature. However, the long preamble problem with acoustic modems revealed in real experiments brings new challenges to underwater MAC design. The overhead of control messages in handshaking-based protocols becomes significant due to the long preamble in underwater acoustic modems. To address this problem, we advocate the receiver-initiated handshaking method with parallel reservation to improve the handshaking efficiency. Despite some existing works along this direction, the data polling problem is still an open issue. Without knowing the status of senders, the receiver faces two challenges for efficient data polling: when to poll data from the sender and how much data to request. In this paper, we propose a traffic estimation-based receiver-initiated MAC (TERI-MAC) to solve this problem with an adaptive approach. Data polling in TERI-MAC depends on an online approximation of traffic distribution. It estimates the energy efficiency and network latency and starts the data request only when the preferred performance can be achieved. TERI-MAC can achieve a stable energy efficiency with arbitrary network traffic patterns. For traffic estimation, we employ a resampling technique to keep a small computation and memory overhead. The performance of TERI-MAC in terms of energy efficiency, channel utilization, and communication latency is verified in simulations. Our results show that, compared with existing receiver-initiated-based underwater MAC protocols, TERI-MAC can achieve higher energy efficiency at the price of a delay penalty. This confirms the strength of TERI-MAC for delay-tolerant applications.