The Internet of Things (IoT) edge network has connected lots of heterogeneous smart devices, thanks to unmanned aerial vehicles (UAVs) and their groundbreaking emerging applications. Limited computational capacity and energy availability have been major factors hindering the performance of edge user equipment (UE) and IoT devices in IoT edge networks. Besides, the edge base station (BS) with the computation server is allowed massive traffic and is vulnerable to disasters. The UAV is a promising technology that provides aerial base stations (ABSs) to assist the edge network in enhancing the ground network performance, extending network coverage, and offloading computationally intensive tasks from UEs or IoT devices. In this paper, we deploy a clustered multi-UAV to provide computing task offloading and resource allocation services to IoT devices. We propose a multi-agent deep reinforcement learning (MADRL)-based approach to minimize the overall network computation cost while ensuring the quality of service (QoS) requirements of IoT devices or UEs in the IoT network. We formulate our problem as a natural extension of the Markov decision process (MDP) concerning stochastic game, to minimize the long-term computation cost in terms of energy and delay. We consider the stochastic time-varying UAVs’ channel strength and dynamic resource requests to obtain optimal resource allocation policies and computation offloading in aerial to ground (A2G) network infrastructure. Simulation results show that our proposed MADRL method reduces the average costs by 38.643%, and 55.621% and increases the reward by 58.289% and 85.289% compared with the different single agent DRL and heuristic schemes, respectively.

In the fifth-generation (5G) wireless networks, Edge-Internet-of-Things (EIoT) devices are envisioned to generate huge amounts of data. Due to the limitation of computation capacity and battery life of devices, all tasks cannot be processed by these devices. However, mobile-edge computing (MEC) is a very promising solution enabling offloading of tasks to nearby MEC servers to improve quality of service. Also, during emergency situations in areas where network failure exists, unmanned aerial vehicles (UAVs) can be deployed to restore the network by acting as Aerial Base Stations and computational nodes for the edge network. In this article, we consider a central network controller who trains observations and broadcasts the trained data to a multi-UAV cluster network. Each UAV cluster head acts as an agent and autonomously allocates resources to EIoT devices in a decentralized fashion. We propose model-free deep reinforcement learning (DRL)-based collaborative computation offloading and resource allocation (CCORA-DRL) scheme in an aerial to ground (A2G) network for emergency situations, which can control the continuous action space. Each agent learns efficient computation offloading policies independently in the network and checks the statuses of the UAVs through Jain’s Fairness index. The objective is minimizing task execution delay and energy consumption and acquiring an efficient solution by adaptive learning from the dynamic A2G network. Simulation results reveal that our scheme through deep deterministic policy gradient, effectively learns the optimal policy, outperforming A3C, deep $Q$ -network and greedy-based offloading for local computation in stochastic dynamic environments.

Electric Vehicles (EVs) have emerged as one of the most promising solutions for reducing carbon emissions in smart cities. However, due to the limited battery life of EVs and the scarcity of charging stations, EV drivers are not willing to travel long distances. Thus, blockchain-enabled energy trading (BET) has lately been used to securely share energy among EVs via wireless power transfer (WPT) technology. Blockchain is used to ensure the security and privacy of transactions between untrustworthy EVs in the WPT process. Nevertheless, previous works on BET have relied on existing consensus mechanisms built on the requirements of the cryptocurrency systems. These consensus mechanisms have faced significant challenges in maintaining high reliability, throughput, low latency, and network scalability in V2V energy trading that requires real-time services. To address these issues, we propose a new consensus mechanism that leverages the benefits of Practical Byzantine Fault Tolerance (PBFT) and Proof of Reputation (PoR) called PBFT-based PoR (PPoR). The energy trading process runs in a clustered vehicular network, where validator selection, block generation, and consensus processes are performed in each cluster. We adopt an incentive mechanism based on a Stackelberg game model to optimize the utility of sellers, buyers, and validator nodes, which motivates honest and cooperative nodes. The simulation results show that the proposed scheme reduces buyers’ costs by 21.1% while increasing the utility of sellers by 18%. Moreover, compared to benchmarks, the proposed scheme reduces the transaction processing delay and increases the throughput by more than 47.1% and 15.7%, respectively.

The elastic reconstruction of 5G network services is expected to provide the capability of network slice orchestration to access the network on demand, guarantee service experience on demand, and construct services on demand as well as to construct basic network services with lower costs. It is challenging to have different applications served independently with a proper resource allocation mechanism according to their own requirements. In this paper, we propose a dynamic resource reservation and deep reinforcement learning-based autonomous virtual resource slicing framework for the next generation radio access network. The infrastructure provider periodically reserves the unused resource to the virtual networks based on their ratio of minimum resource requirements. Then, the virtual networks autonomously control their resource amount using deep reinforcement learning based on the average quality of service utility and resource utilization of users. With the defined framework in this paper, virtual operators can customize their own utility function and objective function based on their own requirements. The simulation results show the performances on convergence rate, resource utilization, and satisfaction of the virtual networks.

Network slicing has been introduced in fifth-generation (5G) systems to satisfy requirements of diverse applications from various service providers operating on a common shared infrastructure. However, heterogeneous characteristics of slices have not been widely explored. In this paper, we investigate dynamic network slicing strategies with mixed traffics in virtualized radio access network (RAN). Considering versatile users’ quality of service (QoS) requirements on transmission delay and rate, coarse resource provisioning scheme and deep reinforcement learning-based autonomous slicing refinement algorithm are proposed. Then, a shape-based heuristic algorithm for user resource customization is devised to improve resource utilization and QoS satisfaction. In principle, the DQN algorithm allocates only the necessary resource to slices to satisfy users’ QoS requirements. For fairness in comparison, we reserve all the unused resources back to the slices. In case there is a sudden change in user population in one slice, the algorithm provides isolation. To validate the advantage, system-level simulations are conducted. The results show that the proposed algorithm balances the satisfaction up to about 100% and resource utilization up to 80% against state-of-the-art solutions. The proposed algorithm also improves the performance of slices in mixed traffics against state-of-the-art benchmarks, which fail to balance satisfaction and resource utilization in some slices.

Network slicing is one of the main enablers of the fifth-generation (5G) cellular network. However, it is susceptible to security threats such as distributed denial of service (DDoS) attacks. A DDoS attack on a slice could lead to the exhaustion of available common resources and a breach of the availability of resources on the slices. Recent works such as statistical, machine learning and cryptography techniques are limited by the requirement to define thresholds, feature engineering constraints and computation overload, respectively. In this letter, we propose DeepSecure, a framework based on a Long Short Term Memory deep learning technique that detects user equipment (UE) network traffic as DDoS attack or normal traffic and assigns an appropriate slice to a legitimate UE request. We compared our work with existing machine learning and deep learning techniques used in the literature. Experiment results showed that our proposed framework performed better in detecting DDoS attacks with an accuracy of 99.970% and predicting the appropriate slice requested by legitimate UE with an accuracy of 98.798%.