14 September 2018 in IEEE Transactions on Signal Processing
Spectrum sensing in a large-scale heterogeneous network is very challenging as it usually requires a large number of static secondary users (SUs) to obtain the global spectrum states. To tackle this problem, in this paper, we propose a new framework based on Bayesian machine learning. We exploit the mobility of multiple SUs to simultaneously collect spectrum sensing data and cooperatively derive the global spectrum states. We first develop a novel non-parametric Bayesian learning model, referred to as beta process (BP) sticky hidden Markov model (SHMM), to capture the spatial-temporal correlation in the collected spectrum data, where the SHMM models the latent statistical correlation within each mobile SU's time series data, while the BP realizes the cooperation among multiple SUs. Bayesian inference is then carried out to automatically infer the heterogeneous spectrum states. Based on the inference results, we also develop a new algorithm with a refinement mechanism to predict the spectrum availability, which enables a newly joining SU to immediately access the unoccupied frequency band without sensing. Simulation results show that the proposed framework can significantly improve spectrum sensing performance compared with the existing spectrum sensing techniques.
22 January 2021 in IEEE Communications Letters
In network slicing, dynamic resource allocation is the key to network performance optimization. Deep reinforcement learning (DRL) is a promising method to exploit the dynamic features of network slicing by interacting with the environment. However, the existing DRL-based resource allocation solutions can only handle a discrete action space. In this letter, we tackle a general DRL-based resource allocation problem which considers a mixed action space including both discrete channel allocation and continuous energy harvesting time division, with the constraints of energy consumption and queue package length. We propose a novel DRL algorithm referred to as constrained discrete-continuous soft actor-critic (CDC-SAC) by redesigning the network architecture and policy learning process. Simulation results show that the proposed algorithm can achieve a significant performance improvement in terms of the total throughput with the strict constraints guarantee.
22 December 2020 in IEEE Transactions on Signal Processing
Mobile-edge computing (MEC) offloads computational tasks from wireless devices to network edge, and enables real-time information transmission and computing. Most existing work concerns a small-scale synchronous MEC system. In this paper, we focus on a large-scale asynchronous MEC system with random task arrivals, distinct workloads, and diverse deadlines. We formulate the offloading policy design as a restless multi-armed bandit (RMAB) to maximize the total discounted reward over the time horizon. However, the formulated RMAB is related to a PSPACE-hard sequential decision-making problem, which is intractable. To address this issue, by exploiting the Whittle index (WI) theory, we rigorously establish the WI indexability and derive a scalable closed-form solution. Consequently, in our WI policy, each user only needs to calculate its WI and report it to the BS, and the users with the highest indices are selected for task offloading. Furthermore, when the task completion ratio becomes the focus, the shorter slack time less remaining workload (STLW) priority rule is introduced into the WI policy for performance improvement. When the knowledge of user offloading energy consumption is not available prior to the offloading, we develop Bayesian learning-enabled WI policies, including maximum likelihood estimation, Bayesian learning with conjugate prior, and prior-swapping techniques. Simulation results show that the proposed policies significantly outperform the other existing policies.