Due to the dynamic nature of connectivity in terrestrial vehicular networks, it is of great benefit to deploy unmanned aerial vehicles (UAVs) in these networks to act as relays. As a result, a remarkable number of studies have exploited UAVs to bridge the communication gaps between terrestrial vehicles, and sometimes despite their unoptimized mobility, their restricted communication coverage, and their limited energy resources. However, it was noted that for an intermittently connected vehicular network, UAVs could not cover all sparse areas all the time. Even worse, when deploying enough UAVs to cover all these areas, the probability of inter-UAV collisions increases, and it will be complex to control their movements efficiently. Consequently, it is required to dispatch an organized and intelligent group of UAVs to perform communication relays in the long term while keeping their connectivity, minimizing their average energy consumption, and providing an efficient coverage strategy. To meet these requirements, we propose a deep reinforcement learning (DRL) framework, called DISCOUNT (Dispatch of UAVs for Urban VANETs). Extensive simulations have been conducted to evaluate the performance of the proposed framework. It has been shown that the proposed framework significantly outperforms two commonly-used baseline techniques and some reinforcement learning methods in terms of energy consumption, coverage, and routing performances.

Anomaly detection is one of the biggest issues of security in the Industrial Internet of Things (IIoT) due to the increase in cyber attack dangers for distributed devices and critical infrastructure networks. To face these challenges, the Intrusion Detection System (IDS) is suggested as a robust mechanism to protect and monitor malicious activities in IIoT networks. In this work, we suggest a new mechanism to improve the efficiency and robustness of the IDS system using Distributional Reinforcement Learning (DRL) and the Generative Adversarial Network (GAN). We aim to develop realistic and equilibrated distribution for a given feature set using artificial data in order to overcome the issue of data imbalance. We show how the GAN can efficiently assist the distributional RL-based-IDS in enhancing the detection of minority attacks. To assess the taxonomy of our approach, we verified the effectiveness of our algorithm by using the Distributed Smart Space Orchestration System (DS2OS) dataset. The performance of the normal DRL and DRL-GAN models in binary and multiclass classifications was evaluated based on anomaly detection datasets. The proposed models outperformed the normal DRL in the standard metrics of accuracy, precision, recall, and F1 score. We demonstrated that the GAN introduced in the training process of DRL with the aim of improving the detection of a specific class of data achieves the best results.

Wireless Sensor Network (WSN) applications have become more and more attractive with the miniaturization of circuits and the large variety of sensors. The different application domains, especially critical fields of WSN use, make the reliability of data acquisition and communication a hot research field that must be tackled efficiently. Indeed, the quality of largely used, cheap-cost wireless sensors and their scarce energy supply support these reliability challenges that lead to data loss or corruption. For solving this problem, the conception of a reliability mechanism that detects these shortcomings and recovers to them becomes necessary. In this article, we present a survey on existing reliability protocols conceived especially for WSNs due to their special features. The deep classification and discussion in this study allow for understanding the pros and cons of state-of-the-art works in order to enhance the existing schemes and fill the gaps. We have classified the works according to the required level of reliability, the manner to identify the origins of the lack of reliability, and the control to recover this lack of reliability. Across the discussion along this study, we deduce that the cross-layer design between MAC, routing, and transport layers presents a good concept to efficiently overcome the different reliability holes.

Thanks to recent enormous advance of mobile and vehicular technology, it is highly expected that Unmanned Aerial Vehicles (UAVs) are used for various applications in smart cities as well as the number of flying UAVs in the sky is increasing explosively. However, such a proliferation of UAVs accompanies additional critical issues to be considered for efficient, secure use of UAVs. One of important issues should be a privacy of people. When the UAVs fly to perform specific objectives, minimizing movements of UAVs is an important issue to minimize mission completion time and to maximize the network lifetime of UAVs. To do so, one intuitive solution is that UAVs may pass through private area of citizens whereas people do not want any penetration into own area without a permission. For those situations, we may take into account a compromise so that people can decide whether they give a differential, temporal permission for each UAV to access their areas depending on specific benefits by use of UAVs or emergent situation for public safety. In this study, we introduce a framework for privacy preserving movements of UAVs with differential UAVs' permissions given by citizens, which is called as UDiPP (UAVs' Differential Privacy Preserving movements). Then, using ILP, we formally define a problem whose objective is to minimize total movements of UAVs with preserving privacy of citizens. To solve the problem, we propose a novel approach with a creation of UDiPP graph and then evaluate its performance through extensive simulations.

LoRaWAN attracted lots of attention with its capacity for large device numbers, long-range, and low-power consumption. In order to simplify the transmission procedure, a pure Aloha protocol is implemented into its MAC layer. However, as the number of connected devices to the base station increases, the devices’ transmission parameters allocation becomes a vital issue related to network performance. This research contributes to the decentralized dynamic spreading factor (SF) allocation strategies during transmission by proposing a score table-based evaluation and parameters surfing (STEPS) approach. STEPS is a reinforcement learning-based method that evaluates and changes the parameters based on probability and score tables. It provides a nondeterministic parameter selection method by updating the table while transmitting. Some variants of STEPS with different algorithms are proposed. Moreover, an estimation-based initialization is proposed to improve learning performance. Simulations and statistical tests are carried out with MULANE, a lightweight LoRaWAN Simulator developed in our previous work. The results show that the estimation has a high confidence level. Compared with the baseline methods, the proposed methods reduce energy consumption by 24%–27% in different numbers of nodes. For bi-directional transmission, the proposed methods increase the 18% network throughput in a small number of nodes and 33% in a large number of nodes. Moreover, the proposed methods provide a framework of decentralized parameter allocation, which gives the extendability of this work.

Optimization of energy usage in wireless sensor networks (WSN) has been an active research field for the last decades and various approaches have been explored. In fact, A well designed energy consumption model is the foundation for developing and evaluating a power management scheme in network of energy constrained devices such as: WSN. We are interested in developing optimal centralized power control policies for energy harvesting wireless multimedia sensor networks (WMSN) equipped with photovoltaic cells. We propose a new complete information Markov decision process model to characterize sensor's battery discharge/recharge process and inspect the structural properties of optimal transmit policies.