16 November 2022 in IEEE Transactions on Communications
With the ubiquitous deployment of a massive number of Internet-of-Things (IoT) devices, the satellite-aerial networks are becoming a promising candidate to provide flexible and seamless service for IoT applications. Concerning about the spectrum scarcity issue, we present a cognitive satellite-aerial network where the multiple unmanned aerial vehicles (UAVs) can share spectrum with the satellite without interfering with satellite communications. To further improve spectral efficiency, non-orthogonal multiple access (NOMA) technique is adopted in this network. Considering the delay-sensitive quality-of-service (QoS) requirement, a joint trajectory and power optimization problem is formulated to minimize the total transmission latency over a long-term task period. In order to reduce the computational complexity and ease the burden of information exchange by using the centralized DRL methods, we propose a multi-agent deep deterministic policy gradient (MADDPG) based algorithm which adopts the framework of centralized training with decentralized execution to solve the sophisticated problem. The simulation results show the proposed algorithm can achieve satisfactory performance through joint trajectory control and power allocation for the UAVs compared with other methods.
13 January 2022 in IEEE Transactions on Cognitive Communications and Networking
Spectrum scarcity and energy limitation are becoming two critical issues in designing Internet of Things (IoT). As two promising technologies, cognitive radio (CR) and radio frequency (RF) energy harvesting can be used together to improve both energy and spectral efficiency. In this paper, an optimal transmission problem in a cognitive IoT (CIoT) with RF energy harvesting capability is investigated, where the optimization problem is formulated as a Markov decision process (MDP) without any priori-knowledge. Considering that the channel activity states of primary user network (PUN), RF energy arrival process and channel information are not available in advance, a deep reinforcement learning (DRL) based deep deterministic policy gradient (DDPG) algorithm is proposed to deal with the dynamic uplink access, working mode selection and continuous power allocation to maximize a long term uplink throughput. The simulation results show that the proposed algorithm is valid and efficient to achieve better performances when compared with deep ${Q}$ -network (DQN) based, myopic and random algorithms.
6 December 2023 in IEEE Transactions on Wireless Communications
In this paper, we present a hybrid ambient backscatter communication (ABC) assisted framework for radio frequency (RF) powered cognitive radio networks (CRNs). In these CRNs, the secondary users (SUs) can actively transmit data when the primary user network (PUN) is idle, and harvest energy from the primary signal and transmit their own information over the primary signal when the PUN is busy. The proposed CRNs adopt non-orthogonal multiple access (NOMA) technique to further improve spectral efficiency. Considering two different spectrum sharing paradigms of the PUN and the SUs, we formulate two optimization problems by two Markov decision processes (MDPs) to maximize our long term throughputs for both underlay-interweave and overlay-interweave scenarios. We propose a deep reinforcement learning (DRL) based optimization algorithm, i.e., a deep deterministic policy gradient (DDPG)-based joint reflection coefficients adjustment and resource allocation (JCARA) algorithm, to solve these two non-convex problems under the two constructed MDPs without the non-causal and the statistical information about the dynamic environment a-priori. For the underlay-interweave scenario, the proposed JCARA algorithm jointly optimizes the transmit power and the reflection coefficients of the SUs, while for the overlay-interweave scenario it optimizes the above two variables plus time resource simultaneously. Simulation results clearly show the higher throughput performance of this proposed algorithm for the proposed CRNs in the comparison with other algorithms.
22 August 2025 in IEEE Internet of Things Journal
This study focuses on a free space optical (FSO)/radio frequency (RF) satellite-UAV-terrestrial integrated network (SUTIN) to overcome limitations of traditional RF SUTIN on spectrum scarcity and security. Traditional resource allocations for FSO/RF SUTIN often prioritize transmission efficiency while neglecting reliability. Moreover, in this network, atmospheric turbulence, pointing errors, and multipath fading can lead to unreliable data transmission. To resolve these problems, we propose a hybrid FSO/RF SUTIN based on adaptive modulation and coding schemes (MCS) and resource allocation algorithm to ensure both transmission efficiency and reliability with imperfect channel state information (CSI) and random channel fading. A joint optimization problem is formulated, incorporating MCS mode selection, power allocation, and UAV trajectory optimization, with frame error rate (FER) and CSI uncertainty constraints to maximize system throughput. To solve this complicated high dimension optimization problem, we present mathematical mapping models between the optimization variables and communication performance parameters (the average transmission rate and FER) with imperfect CSI and random fading for both FSO and RF link. According to these mapping models, a robust solution based on deep deterministic policy gradient is proposed, integrating decoupling and user clustering techniques to reduce the computational complexity. The simulation results demonstrate that this proposed FSO/RF SUTIN and the corresponding algorithm can improve system throughput and guarantee transmission reliability.
1 January 2026 in IEEE Internet of Things Journal
27 September 2017 in Optoelectronics Letters