18 June 2021 in IEEE Transactions on Communications
Network slicing is a key paradigm in 5G and is expected to be inherited in future 6G networks for the concurrent provisioning of diverse quality of service (QoS). Unfortunately, effective slicing of Radio Access Networks (RAN) is still challenging due to time-varying network situations. This paper proposes a new intelligent RAN slicing strategy with two-layered control granularity, which aims at maximizing both the long-term QoS of services and spectrum efficiency (SE) of slices. The proposed method consists of an upper-level controller to ensure the QoS performance, which enforces loose control by performing adaptive slice configuration according to the long-term dynamics of service traffic. The lower-level controller is to improve SE of slices, by tightly scheduling radio resources to users at the small time-scale. To realize the proposed RAN slicing strategy, we propose a model-free deep reinforcement learning (DRL) framework, which is a hierarchical structure that collaboratively integrating the modified deep deterministic policy gradient (DDPG) and double deep-Q-network algorithm. Specifically, the lower-level control problem is a mixed-integer stochastic optimization problem with multiple constraints. This kind of problem is hard to be directly solved by the exiting DRL algorithms, since it involves searching for the solution in a vast set of mixed-integer action space, which will induce unbearable computational complexity. Thus, we propose a novel action space reducing approach, embedding the convex optimization tools into the DDPG algorithm, to speed up the lower-level control. Furthermore, simulation results confirm the effectiveness of our proposed intelligent RAN slicing scheme.
27 October 2020 in IEEE Transactions on Wireless Communications
The ever-growing wireless applications and their diverse Quality of Service (QoS) requirements bring the challenge of tailored QoS provisioning with limited radio resources in future cellular networks. While resource constraint is ubiquitous, different communication equipment in cellular networks could experience very different constraints in the multi-dimensional resource domains. To achieve stringent yet diverse QoS with limited resources, a novel multi-dimensional intelligent multiple access (MD-IMA) scheme is proposed in this paper to exploit disparate resource constraints among heterogeneous equipment for 5G beyond and 6G networks. With the assist of real-time data analysis, real-time QoS requirements, and resource availability of the related equipment are first determined in the proposed MD-IMA. Based on this, multiple access (MA) scheme is then intelligently adapted accordingly for each equipment in multi-dimensional resource domain to maximize the overall system requirement with operational constraints. The resource allocation in the MD-IMA system is further formulated as an optimization problem. To solve this non-convexity optimization of high computational complexity, the overall optimization is divided into several sub-problems and a joint optimization algorithm is adopted. Simulation results demonstrate the system energy efficiency performance gain of proposed MD-IMA over traditional MA is around 15% – 18%.
9 November 2020 in IEEE Journal on Selected Areas in Communications
To meet the ever-increasing communication services with diverse requirements, situation-aware intelligent utilization of multi-dimensional communication resources is becoming essential. In this paper, considering a time-division-duplex downlink cellular scenario, a deep learning-based framework for multi-dimensional intelligent multiple access (MD-IMA) scheme is developed for beyond 5G wireless networks to meet the real-time and diverse quality of service (QoS) requirements by fully utilizing the available radio resources in heterogeneous domains. To achieve intelligent operation of MD-IMA, the proposed deep learning scheme is a convergence of long short term memory (LSTM) and deep reinforcement learning (DRL). Specifically, a LSTM neural network is used to predict the long-term network dynamics and inference changes in QoS requirements of the MD-IMA. Meanwhile, a deterministic policy gradient (DDPG) algorithm, a model-free DRL technique, is adopted to optimize the multi-dimensional radio resource allocation in real-time by dynamically following the fluctuations of the network situation. With the aid of the DDPG algorithm, radio resource management for MD-IMA can be achieved efficiently with reduced processing latency as compared to the conventional model-based approaches. Furthermore, the effectiveness of our proposed deep learning framework for MD-IMA is validated through real-world cellular traffic data-sets. The experimental results demonstrate that the proposed scheme can outperform state-of-the-art algorithms.
6 September 2022 in IEEE Transactions on Communications
As one of the key technologies in 5G networks, Carrier Aggregation (CA) is studied in this paper. In CA, Component Carriers (CCs) can be activated and deactivated depending on multiple factors, e.g., energy consumption and Quality of Service (QoS) demand of users. We propose CC management strategies where each User Equipment (UE) minimizes its average delay and at the same time minimizes its power consumption while considering that CCs can be activated and deactivated only at certain times, as in real-world CA implementations. We first model the problem as a centralized multi-objective optimum CC management problem. Since centralized approaches would impose a large overhead on the system, we then develop a semi-distributed solution by modeling the problem as a stochastic game and propose a multi-agent Double Deep Q-Network (DDQN) based CC management algorithm to solve the stochastic game. We finally compare the proposed approaches with single CC activation and all-CC activation baseline schemes. Simulation results show that our proposed algorithms outperform the all-CC algorithm in terms of UE power consumption and have the capability of transmitting a number of bits with delay close to the all-CC scheme. Meanwhile, our DDQN-based algorithm decreases the UE power consumption by about 20% with respect to the all-CC scheme.
25 October 2024 in IEEE Networking Letters
Beam management is an important technique to improve signal strength and reduce interference in wireless communication systems. Recently, there has been increasing interest in using diverse sensing modalities for beam management. However, it remains a big challenge to process multi-modal data efficiently and extract useful information. On the other hand, the recently emerging multi-modal transformer (MMT) is a promising technique that can process multi-modal data by capturing long-range dependencies. While MMT is highly effective in handling multi-modal data and providing robust beam management, integrating reinforcement learning (RL) further enhances their adaptability in dynamic environments. In this work, we propose a two-step beam management method by combining MMT with RL for dynamic beam index prediction. In the first step, we divide available beam indices into several groups and leverage MMT to process diverse data modalities to predict the optimal beam group. In the second step, we employ RL for fast beam decision-making within each group, which in return maximizes throughput. Our proposed framework is tested on a 6G dataset. In this testing scenario, it achieves higher beam prediction accuracy and system throughput compared to both the MMT-only based method and the RL-only based method.
28 April 2021 in IEEE Networking Letters
In this letter, we consider 5G networks with Carrier Aggregation (CA). Our aim is to jointly select Component Carriers (CCs) and allocate Resource Blocks (RBs) such that total user throughput is maximized while user power consumption is minimized and Quality of Service (QoS) requirements are met. We formulate the User Equipment (UE) throughput and power consumption in terms of CC and RB indicators and propose a multi-objective optimization problem. Simulation results show that the proposed scheme outperforms the compared techniques by providing approximately 200mW reduction in power consumption while increasing the throughput by 2.7 times for users under short delay constraint.