31 October 2023 in IEEE Transactions on Mobile Computing
Network slicing is a key enabler of 5G and beyond networks to satisfy the diverse quality of service (QoS) requirements of different services simultaneously. In network slicing, radio access network (RAN) slicing is essential to establish a functional network slice by connecting mobile devices and mapping virtualized resource units to different slices. This requires a highly efficient resource allocation scheme to maximize resource utilization efficiency and meet the diverse QoS requirements. In this paper, we propose a dynamic RAN slicing model that incorporates multiple distributions to accommodate different user request types and diverse priorities among traffic types in the same slice, where the total available resources are dynamically changing over time. We formulate resource allocation as a time-sequential dynamic optimization problem that takes into account system stability, resource limitation, different timescales, long-term system performance, and user priority. We propose a deep reinforcement learning-based (DRL-based) approach referred to as prediction-aided weighted DRL (PW-DRL) to online infer the power allocation and user acceptance decisions that can maximize a predefined reward function. Additionally, a prediction network is formulated to capture the correlation between current and future states. Simulation results validate that our proposed PW-DRL significantly outperforms state-of-the-art approaches by achieving the highest long-term reward and fastest convergence.
14 September 2018 in IEEE Transactions on Signal Processing
Spectrum sensing in a large-scale heterogeneous network is very challenging as it usually requires a large number of static secondary users (SUs) to obtain the global spectrum states. To tackle this problem, in this paper, we propose a new framework based on Bayesian machine learning. We exploit the mobility of multiple SUs to simultaneously collect spectrum sensing data and cooperatively derive the global spectrum states. We first develop a novel non-parametric Bayesian learning model, referred to as beta process (BP) sticky hidden Markov model (SHMM), to capture the spatial-temporal correlation in the collected spectrum data, where the SHMM models the latent statistical correlation within each mobile SU's time series data, while the BP realizes the cooperation among multiple SUs. Bayesian inference is then carried out to automatically infer the heterogeneous spectrum states. Based on the inference results, we also develop a new algorithm with a refinement mechanism to predict the spectrum availability, which enables a newly joining SU to immediately access the unoccupied frequency band without sensing. Simulation results show that the proposed framework can significantly improve spectrum sensing performance compared with the existing spectrum sensing techniques.
26 September 2019 in IEEE Transactions on Vehicular Technology
This paper addresses the spectrum sensing problem in an orthogonal frequency-division multiplexing (OFDM) system based on machine learning. To adapt to signal-to-noise ratio (SNR) variations, we first formulate the sensing problem into a novel SNR-related multi-class classification problem. Then, we train a naive Bayes classifier (NBC), and propose a class-reduction assisted prediction method to reduce spectrum sensing time. We derive the performance bounds by translating the Bayes error rate into spectrum sensing error rate. Compared with the conventional methods, the proposed method is shown by simulation to achieve higher spectrum sensing accuracy, in particular at critical areas of low SNRs. It offers a potential solution to the hidden node problem.
22 January 2021 in IEEE Communications Letters
In network slicing, dynamic resource allocation is the key to network performance optimization. Deep reinforcement learning (DRL) is a promising method to exploit the dynamic features of network slicing by interacting with the environment. However, the existing DRL-based resource allocation solutions can only handle a discrete action space. In this letter, we tackle a general DRL-based resource allocation problem which considers a mixed action space including both discrete channel allocation and continuous energy harvesting time division, with the constraints of energy consumption and queue package length. We propose a novel DRL algorithm referred to as constrained discrete-continuous soft actor-critic (CDC-SAC) by redesigning the network architecture and policy learning process. Simulation results show that the proposed algorithm can achieve a significant performance improvement in terms of the total throughput with the strict constraints guarantee.
16 September 2024 in IEEE Journal on Selected Areas in Communications
Space-Air-Ground integrated network (SAGIN) is a crucial component of the 6G, enabling global and seamless communication coverage. This multi-layered communication system integrates space, air, and terrestrial segments, each with computational capability, and also serves as a ubiquitous computing platform. An efficient task offloading and resource allocation scheme is key in SAGIN to maximize resource utilization efficiency, meeting the stringent quality of service (QoS) requirements for different service types. In this paper, we introduce a dynamic SAGIN model featuring diverse antenna configurations, two timescale types, different channel models for each segment, and dual service types.We formulate a problem of sequential decision-making task offloading and resource allocation. Our proposed solution is an innovative online approach referred to as graphic DRL (GDRL). This approach utilizes a graph neural network (GNN)-based feature extraction network to identify the inherent dependencies within the graphical structure of the states. We design an action mapping network with an encoding scheme for end-to-end generation of task offloading and resource allocation decisions. Additionally, we incorporate meta-learning into GDRL to swiftly adapt to rapid changes in key parameters of the SAGIN environment, significantly reducing online deployment complexity. Simulation results validate that our proposed GDRL significantly outperforms state-of-the-art DRL approaches by achieving the highest reward and lowest overall latency.