4 October 2018 in IEEE Access
On the basis of ${\mathcal{ H_\infty }}$ control approach instead of using the conventional optimization method, we present a distributed power allocation algorithm for a cognitive radio network (CRN), where underlying secondary users (SUs) share same licensed spectrum with primary users (PUs) and the channel gain is time-varying between two time slots. Based on the target signal-to-interference-plus-noise ratio (target-SINR) tracking power control (TPC) algorithm in the conventional network and the dynamic description of the channel gain fluctuation in the CRN as a first-order Markov model, we formulate the power allocation in the CRN into a state-space system model with exogenous input. In this time, the core for us becomes to design a ${\mathcal{ H_\infty }}$ state feedback controller obtained by solving a linear matrix inequality (LMI) for this system to realize power allocation for SUs. The SINR requirement of SUs and the interference temperature (IT) constraint of all PUs can be guaranteed. According to this controller design principle, we also give a ${\mathcal{ H_\infty }}$ delay-independent state feedback controller to treat time-delay for the protection of the communication performance. Simulation results demonstrate the validity, effectiveness, and advantages of this approach compared with the algorithms obtained by the optimization theory for the power allocation in CRNs.
24 August 2022 in IEEE Transactions on Network Science and Engineering
Without control center assistance, it is difficult that users take traffic loads of whole Ad Hoc networks into account to utilize wireless resources collaboratively. Considering influence of load differences between users on resource allocation can be learned in advance, we construct a framework of load-aware distributed resource allocation for multiple frequencies time division multiple access (MF-TDMA) wireless Ad Hoc networks based on multi-agent deep reinforcement learning (DRL) with centralized training and distributed execution. In this framework, each user is regarded as an agent and the wireless transmission resources are partitioned into time-frequency resource blocks (RBs). Each user reserves RBs by sending network manager operational packet (NMOP) in turn at rotating broadcast stage (RBS) and how many RBs each user reserves depends on the corresponding agent's decision. The main innovative point of our work is achieving a result of wireless resource allocation that balance load degree of all users on the whole network in a distributed manner. With defining a novel network utility parameterized by traffic loads (termed timely capacity), we first model the resource allocation as an optimization problem for timely capacity sum maximization. Using the optimization objective as common reward function to train neural networks of all agents in an offline simulation environment, we obtain the distributed algorithm based on multi-agent DRL. Experimental results demonstrate that with the neural networks trained by the designed reward and enough training, users can successfully learn to collaboratively improve network utility in distributed implementation.
23 August 2023 in IEEE Transactions on Cognitive Communications and Networking
Real-time information exchange on traffic and channel selection results among users in dynamic spectrum access (DSA) system consumes scarce spectrum resources. However, it is difficult to avoid collision and improve system-wide global utility simultaneously without assistance of these information in a distributed way. To solve this problem, we propose a multi-agent deep reinforcement learning (RL) based traffic priority-aware multi-user distributed DSA scheme for a multiple orthogonal channels scenario. Different from the conventional approaches for throughput sum maximization, we maximize a total network utility parameterized by the situation of each user’s traffic buffer queue. This scheme includes off-line centralized training and distributed execution. The deep Q-learning neural network (DQN) of each user is trained by an offline simulator with global information to learn near-optimal channel selection policies from the transition history. The input of DQN requires only user’s local observation to ensure that the scheme based on the trained DQNs can be executed in a distributed way. Simulation results show that the proposed scheme compared with benchmark algorithms can achieve about 90% or more of performance of Genie-aided algorithm based on global information, and is much better than random-type algorithms.
17 July 2018 in IEEE Access
We present a distributed power allocation algorithm for a cognitive radio network (CRN), where the underlying secondary users (SUs) share the same licensed spectrum with the primary users (PUs). Based on the target signal-to-interference-plus-noise ratio (target-SINR) tracking power control (TPC) algorithm in a conventional network, the power allocation problem is modeled as a state-space model with exogenous input that includes varying channel state information (CSI) and some parameter measurement errors. This power allocation algorithm is actually a state feedback controller from the linear quadratic Gaussian (LQG) solution, which is used to guarantee the SINR requirement of the SUs and keep the interference temperature (IT) constraint of all PUs under a given threshold. Considering the quality of service (QoS) of the SU, an adaptive control weight and a switching safety margin of the IT threshold are introduced in this algorithm to achieve better performance. Analysis and simulation results show the effectiveness and advantages of the proposed power allocation scheme, designed based on control theory, compared with those of other traditional algorithms designed based on optimization.