Devices used in Internet of Things (IoT) networks continue to perform sensing, gathering, modifying, and forwarding data. Since IoT networks have a lot of participants, mitigating and reducing collisions among the participants becomes an essential requirement for the Medium Access Control (MAC) protocols to increase system performance. A collision occurs in wireless channel when two or more nodes try to access the channel at the same time. In this paper, a reinforcement learning-based MAC protocol was proposed to provide high throughput and alleviate the collision problem. A collaboratively predicted Q-value was proposed for nodes to update their value functions by using communications trial information of other nodes. Our proposed protocol was confirmed by intensive system level simulations that it can reduce convergence time in 34.1% compared to the conventional Q-learning-based MAC protocol.
Given the ever-increasing interest in the Internet of Underwater Things (IoUT), various studies are ongoing to solve some of the practical problems affecting the development of underwater wireless communication. The main problems are related to the use of acoustic waves in a water medium, in which extremely high propagation loss and drastic channel fluctuation are common. On the basis of hands-on experience and measurements made in real underwater environments, the conventional Adaptive Modulation and Coding (AMC), which uses the high correlation between SNR (Signal to Noise Ratio) and BER (Bit Error Rate), might not be affordable in underwater environments because the normal correlation between SNR and BER almost disappears altogether. This work therefore collectively takes into account multiple quality factors of communication at the same time by creating, analysing and validating the machine learning model to predict the most adequate communication parameters to solve the problem. The dataset of underwater wireless communication used in the learning models was obtained from measurements made in a real underwater environment near the Gulf of Incheon, South Korea, using a practical testbed designed and implemented by the authors. The estimated network throughput based on the communications parameters predicted using the machine learning models was enhanced by up to 25% compared with the conventional handcraft method.
Lithium-ion batteries are widely used in many applications due to their high energy density, high efficiency, and excellent cycle ability. Once an unknown Li-ion battery is reusable, it is important to measure its lifetime and state of health. The most favorable measurement method is the cycle test, which is accurate but time- and capacity-consuming. In this study, instead of a cycle test, we present an empirical model based on the C-rate test to understand the state of health of the battery in a short time. As a result, we show that the partially accelerated charge/discharge condition of the Li-ion battery is highly effective for the degradation of battery capacity, even when half of the charge/discharge conditions are the same. This observation provides a measurable method for predicting battery reuse and future capacity degradation.
19 September 2022 in IEEE Wireless Communications Letters
Inter-cell interference in dense cellular networks causes severe network performance degradation because the base stations (BSs) and devices cannot fully utilize all the network status information in practice. Reinforcement learning enables the wireless network to find an effective transmission strategy with limited information and the network entities to operate in a distributed manner. We herein propose a medium access algorithm based on multi-agent reinforcement learning by exchanging knowledge of transmissions from other cells. Specifically, a state mapping function between the device and the BS and a transfer value iteration are proposed. The BS transfers reward processes derived from inter-cell interference so that the partially observed medium of devices is expanded to monitor entire networks. Extensive simulation results show that simple knowledge transfer can help performance improvement in terms of network stability and throughput.