29 October 2021 in IEEE Transactions on Network and Service Management
The advent of radio access network (RAN) slicing is envisioned as a new paradigm for accommodating different virtualized networks on a single infrastructure in 5G and beyond. Consequently, infrastructure providers (InPs) desire virtualized networks to share their subleased resources for effective resource management. Nonetheless, security and privacy challenges in the wireless network deter operators from collaborating with one another for resource trading. Lately, blockchain technology has received overwhelming attention for secure resource trading thanks to its security features. This paper proposes a novel hierarchical framework for blockchain-based resource trading among peer-to-peer (P2P) mobile virtual network operators (MVNOs), for autonomous resource slicing in 5G RAN. Specifically, a consortium blockchain network that supports hyperledger smart contract (SC) is deployed to set up secure resource trading among seller and buyer MVNOs. With the aim of designing a fair incentive mechanism, we model the pricing and demand problem of the seller and buyers as a two-stage Stackelberg game, where the seller MVNO is the leader and buyer MVNOs are followers. To achieve a Stackelberg equilibrium (SE) for the formulated game, a dueling deep Q-network (Dueling DQN) scheme is designed to achieve optimal pricing and demand policies for autonomous resource allocation at negotiation interval. Comprehensive simulation results analysis prove that the proposed scheme reduces double spending attacks by 12% in resource trading settings, and maximizes the utilities of players. The proposed scheme also outperforms deep Q-Network (DQN), Q-learning (QL) and greedy algorithm (GA), in terms of slice and system level satisfaction and resource utilization.
17 April 2020 in IEEE Systems Journal
Considering bandwidth-hungry and low latency requirement of vehicular communications applications, we propose a novel dynamic reinforcement learning-based slicing framework and optimization solutions for efficient resource provisioning in virtualized network for D2D-based vehicle-to-vehicle (V2V) communication. The aim is to balance resource utilization and quality of service (QoS) satisfaction levels for multiple slices. The slicing framework is designed as a three-stage layered framework. In the first stage, we propose dynamic deep reinforcement learning-based virtual resource allocation scheme to allocate distinct resources to slices. In the second stage, we aggregate the D2D resource portion of the slice resource for D2D-based V2V communication. In the third stage, due to the computational complexity and signaling overhead of the physical resource allocation, we transform the problem into a convex optimization problem and solve with an alternating direction method of multipliers-based distributed algorithm. Performance results are provided in terms of resource utilization, QoS satisfaction and throughput to show the benefit of integrating resource slices dedicated to supporting interslice D2D-based V2V communication in vehicular network.
2 August 2023 in IEEE/ACM Transactions on Networking
Radio Access Network (RAN) slicing enables resource sharing among multiple tenants and is an essential feature for next-generation mobile networks. Usually, a centralized controller aggregates available resource pools from multiple tenants to increase spectrum availability. In dynamic resource allocation, a tenant could behave strategically by adjusting its preferences based on perceived conditions to maximize its utility. Slice tenants may lie about the resources needed to gain greater utility. Such behavior could lead to poor resource utilization due to excess resources acquired by lying tenants and resource shortages because slice tenants choose not to purchase high-priced resources to save costs. Furthermore, in a scenario with many slice tenants, the centralized controller can become overwhelmed by the number of requests. This, in turn, can lead to slower response times and higher latency, resulting in poor resource utilization and QoS performance of slice tenants. Therefore, this paper proposes a peer-to-peer (P2P) approach to resource trading, where slice tenants communicate directly instead of relying on a centralized orchestrator. This design is motivated by the need for slice tenants to collaborate effectively. We model the interaction between tenants in a Stackelberg multi-leader and multi-follower game and solve the game with multi-agent deep reinforcement learning with an incentive-reward model to achieve the Stackelberg equilibrium. Furthermore, we propose a decentralized resource trading framework by integrating blockchain technology and federated deep reinforcement learning, enabling network tenants to perform inter-slice resource sharing securely. The simulation results show that the proposed mechanism has significant performance improvements over existing implementations.
8 June 2021 in IEEE Systems Journal
With the proliferation of smart mobile devices, there is now an ever increasing craving for higher bandwidth for end user satisfaction. Increasing mobile traffic leads to congestion of backhaul networks. One promising solution to this problem is the mobile edge network and consequently mobile edge caching. There is an emerging paradigm shift toward the use of unmanned aerial vehicles (UAVs) to assist the traditional cellular networks and also to provide connectivity in places where there are no small base stations or faulty ones as a result of some natural disaster such as flooding. Hence, UAVs can be used to assist in content caching as well. This work proposes the use of human centric features, random waypoint user mobility model, and deep reinforcement learning to predict the location of the UAVs and the contents to cache at the UAVs. We formulated our problem as a Markov decision problem (MDP) and proposed a dueling reinforcement learning-based algorithm to solve the MDP problem. Our simulation results prove that our algorithm converges to an optimal solution and performs better than other baseline reinforcement learning algorithms in terms of quality of experience satisfaction, transmit power, and cache resource utilization
7 June 2023 in IEEE Internet of Things Journal
Fifth-generation (5G) wireless networks enable gigabit-per-second data speeds, minimal latency, and reliable Internet of Things (IoT) connectivity. Thus, network slicing (NS) has gained enormous interest due to its ability to improve resource allocation. Due to the exponential growth of IoT data, it is difficult for the infrastructure providers (InPs) to determine the appropriate resource to allocate to mobile virtual network operators (MVNOs). In addition, MVNOs and IoT devices may use self-serving tactics that cause MVNOs to violate service level agreements (SLAs). Therefore, a fundamental problem in NS is capturing the interaction between MVNOs and IoT devices and ensuring efficient use of InP resources. This article proposes a two-tier resource allocation technique for NS involving a monopolistic market between an InP, multiple MVNOs, and IoT devices. First, we model the upper tier problem as a Markov decision problem (MDP) and design a federated deep reinforcement learning-based resource allocation algorithm (FDRL-RA) to explore the optimization solution. At the lower tier, we model a trading market between MVNOs and IoT devices as a two-stage Stackelberg game, where MVNOs set their unit prices and IoT devices set their purchase quantities. We use the backward induction method to analyze the proposed Stackelberg game under a competitive pricing scheme (CPS) and independent pricing scheme (IPS), which ensures high MVNOs’ profit and users’ utility at acceptable levels. Simulation results show that our proposed algorithm converges to the optimal solution and effectively maximizes utility under different pricing schemes while providing a high degree of privacy.
21 October 2024 in IEEE Internet of Things Journal
Sixth generation (6G) networks deploy unmanned aerial vehicles and mobile edge computing to provide collaborative computing and reliable connectivity for resource-limited mobile devices (MDs). However, due to the untrusted and broadcast nature of wireless transmission among communicating MDs and computing resource providers, ensuring the security of resource transactions will be challenging. Blockchain-based resource-sharing systems have been proposed to address security issues. However, these systems use existing consensus mechanisms like Proof-of-Work that consume massive amounts of system resources. In addressing this, some studies attempted to use single-agent deep reinforcement learning (DRL) in leader selection. Nevertheless, these solutions overlooked the intelligence and flexibility of blockchain configuration, and a single-point of failure can cause the system to fail. We propose a multiagent distributed deep deterministic policy gradient (MAD3PG)-assisted consensus mechanism for blockchain-based collaborative resource sharing to address these issues. First, we propose a stochastic game-based incentive-mechanism to encourage consensus nodes to participate in transaction validation. Then, we formulate the optimization problem of node selection and blockchain configuration as a Markov decision process and solve it with the MAD3PG algorithm. With MAD3PG, the agents select consensus nodes based on their experience and available resources and dynamically adjust blockchain settings. The simulation results show that MAD3PG outperforms the benchmarks in maximizing throughput and incentive while minimizing block production latency.