Internet of Things (IoT) devices are increasingly popular due to their wide array of application domains. In IoT networks, sensor nodes are often connected in the form of a mesh topology and deployed in large numbers. Managing these resource-constrained small devices is complex and can lead to high system costs. A number of standardized protocols have been developed to handle the operation of these devices. For example, in the network layer, these small devices cannot run traditional routing mechanisms that require large computing powers and overheads. Instead, routing protocols specifically designed for IoT devices, such as the routing protocol for low-power and lossy networks, provide a more suitable and simple routing mechanism. However, they incur high overheads as the network expands. Meanwhile, reinforcement learning (RL) has proven to be one of the most effective solutions for decision making. RL holds significant potential for its application in IoT device’s communication-related decision making, with the goal of improving performance. In this paper, we explore RL’s potential in IoT devices and discuss a theoretical framework in the context of network layers to stimulate further research. The open issues and challenges are analyzed and discussed in the context of RL and IoT networks for further study.
The next generation of the Internet of Things (IoT) networks is expected to handle a massive scale of sensor deployment with radically heterogeneous traffic applications, which leads to a congested network, calling for new mechanisms to improve network efficiency. Existing protocols are based on simple heuristics mechanisms, whereas the probability of collision is still one of the significant challenges of future IoT networks. The medium access control layer of IEEE 802.15.4 uses a distributed coordination function to determine the efficiency of accessing wireless channels in IoT networks. Similarly, the network layer uses a ranking mechanism to route the packets. The objective of this study was to intelligently utilize the cooperation of multiple communication layers in an IoT network. Recently, Q-learning (QL), a machine learning algorithm, has emerged to solve learning problems in energy and computational-constrained sensor devices. Therefore, we present a QL-based intelligent collision probability inference algorithm to optimize the performance of sensor nodes by utilizing channel collision probability and network layer ranking states with the help of an accumulated reward function. The simulation results showed that the proposed scheme achieved a higher packet reception ratio, produces significantly lower control overheads, and consumed less energy compared to current state-of-the-art mechanisms.
The healthcare Internet of Things (H-IoT) is an interconnection of devices capable of sensing and transmitting information that conveys the status of an individual’s health. The continuous monitoring of an individual’s health for disease diagnosis and early detection is an important application of H-IoT. Ambient assisted living (AAL) entails monitoring a patient’s health to ensure their well-being. However, ensuring a limit on transmission delays is an essential requirement of such monitoring systems. The uplink (UL) transmission during the orthogonal frequency division multiple access (OFDMA) in the wireless local area networks (WLANs) can incur a delay which may not be acceptable for delay-sensitive applications such as H-IoT due to their random nature. Therefore, we propose a UL OFDMA scheduler for the next Wireless Fidelity (Wi-Fi) standard, the IEEE 802.11be, that is compliant with the latency requirements for healthcare applications. The scheduler allocates the channel resources for UL transmission taking into consideration the traffic class or access category. The results demonstrate that the proposed scheduler can achieve the required latency for H-IoT applications. Additionally, the performance in terms of fairness and throughput is also superior to state-of-the-art schedulers.
The Internet of Things (IoT), as a new paradigm of connected things or objects to the Internet, allows us to monitor the environment by collecting data in a wide spatial and temporal window. Especially the utilization of IoT has increased significantly since the development of the Long Range Wide Area Network (LoRaWAN). However, deploying LoRa gateways, maintaining network infrastructure, operational cost, and quality of service are challenging. Helium has emerged as one of the largest networks in terms of coverage for IoT devices to solve such problems. Helium is decentralized, cryptocurrency incentives-based network infrastructure replacing traditional service providers. However, due to network incentives, currently, it contains more hotspots compared to active users. This paper presents our experience and analysis of deploying IoT devices for real-world applications using the Helium network. We present experiences from the IoT device’s deployment for wetland conservation in southern Sweden.
We have seen a promising acceptance of wireless local area networks (WLANs) in our day-to-day communication devices, such as handheld smartphones, tablets, and laptops. Energy preservation plays a vital role in WLAN communication networks. The efficient use of energy remains one of the most substantial challenges to WLAN devices. Several approaches have been proposed by the industrial and institutional researchers to save energy and reduce the overall power consumption of WLAN devices focusing on static/adaptive energy saving methods. However, most of the approaches save energy at the cost of throughput degradation due to either increased sleep-time or reduced number of transmissions. In this paper, we recognize the potentials of reinforcement learning (RL) techniques, such as the Q-learning (QL) model, to enhance the WLAN’s channel reliability for energy saving. QL is one of the RL techniques, which utilizes the accumulated reward of the actions performed in the state-action model. We propose a QL-based energy-saving MAC protocol, named greenMAC protocol. The proposed greenMAC protocol reduces the energy consumption by utilizing accumulated reward value to optimize the channel reliability, which results in reduced channel collision probability of the network. We assess the degrees of channel congestion in collision probability as a reward function for our QL-based greenMAC protocol. The comparative results show that greenMAC protocol achieves enhanced system throughput performance with additional energy savings compared to existing energy-saving mechanisms in WLANs.