Rate-splitting multiple access (RSMA) softly reconciles and decodes the extreme interference by non-orthogonal transmission, which can remarkably solve the spectrum scarcity for future six-generation (6G) low earth orbits (LEO) satellite communication system. In this letter, we investigate the power allocation problem in LEO satellite networks with RSMA mechanism based on the deep reinforcement learning (DRL) technique. Specifically, in order to achieve better RSMA performance, the LEO satellite base station (SBS) has to effectively allocate transmit power to common and private streams, which is very challenging due to the uncertain and limited information of the channel distribution. To solve this problem, a highly-effective proximal policy optimization (PPO) based scheme is further proposed, which enables the LEO SBS to learn an optimal power allocation strategy to maximize the sum rate of the system without knowing any prior information. Simulation results prove that the proposed scheme significantly outperforms the other three baseline schemes in terms of the sum rate metric with low computation complexity.
8 November 2023 in IEEE Transactions on Communications
This paper considers the joint optimization of resource allocation and power control for rate-splitting multiple access (RSMA) based low earth orbits (LEO) satellite-terrestrial networks, where resource sharing between terrestrial and LEO satellite communications is optimized and the LEO satellite serves multiple ground stations (GSs) simultaneously through RSMA technique. Particularly, to make full use of RSMA technique, the LEO satellite needs to appropriately schedule transmitting power to common and private streams. Therefore, the key issue is to jointly optimize the resource allocation and power control to fully utilize the benefits of resource sharing and RSMA. However, the combination of continuous power control and discrete resource allocation becomes the bottleneck for providing an effective solution with limited system information. To deal with this problem, we propose a deep reinforcement learning (DRL)-based framework which jointly employs deep Q-network (DQN) algorithm for discrete resource allocation and proximal policy optimization (PPO) algorithm for continuous power control to maximize a joint objective. Simulation experiments evaluate the performance of the proposed scheme compared with several baseline schemes and the results show the advantages of the proposed scheme.
28 April 2022 in IEEE Wireless Communications Letters
Device-to-device (D2D) communications are envisioned as a critical technology to support future ubiquitous mobile communications applications. However, the requirements of high mobility and low latency seriously constrain the performance improvement of D2D communications. In this letter, we investigate a deep reinforcement learning (DRL) based scheme of power and resource allocation for maximizing the throughput of D2D users (DUEs) and cellular users (CUEs). Based on DRL theory, D2D pairs are defined as distributed multiple agents, which aim to adaptively select the transmission power and resources to ease the co-channel interference without any prior information. Furthermore, a priority sampling based dueling double deep Q-network (PS-D3QN) distributed algorithm is proposed to help agents to learn the predominant features. Simulation results show that the proposed algorithm has higher throughput than the existing DRL algorithms with a strict delay constraint. Particularly, the probability that the agent selects high power decreases with the increase of remaining transmission time, which proves the agents effectively learn and dynamically sense the impacts of the delay constraint.
10 June 2020 in IEEE Transactions on Vehicular Technology
With the development of millimeter-wave (mmWave) technology and vehicle-to-vehicle (V2V) communications, the mmWave vehicular ad hoc networks (VANETs) is envisioned to support rapidly growing number of vehicles. Against this background, each V2V user (VUE) is expected to employ large-scale array to form directional analog beams for improving spatial spectrum reuse, and it is capable of achieving concurrent transmissions from multiple other VUEs simultaneously. However, due to the high dynamics of V2V links, it can be challenging for each VUE to quickly select an effective analog beam. In this paper, we propose a machine learning (ML) approach to achieve an efficient and fast analog beam selection for mmWave V2V communications. Specifically, we first derive the probabilities that multiple V2V transmitters (TXs) serve one VUE to obtain the average sum rate (ASR) for mmWave V2V communications. On that basis, we develop an ML approach to maximize the ASR, whereby the support vector machine (SVM) classifier is utilized for optimizing the analog beam selection. Besides, we further proposed an iteration sequential minimal optimization training algorithm to train data samples of all V2V links, and convergence of the proposed solution is also discussed. Finally, an extensive sample training and simulations are evaluated by Google TensorFlow. The results verified that our proposed ML approach is capable of achieving a higher ASR yet substantially lower computational complexity than traditional solutions based on explicitly estimated channels.
Device-to-device (D2D) communication is regarded as a promising technology to support spectral-efficient Internet of Things (IoT) in beyond fifth-generation (5G) and sixth-generation (6G) networks. This article investigates the spectrum access problem for D2D-assisted cellular networks based on deep reinforcement learning (DRL), which can be applied to both the uplink and downlink scenarios. Specifically, we consider a time-slotted cellular network, where D2D nodes share the cellular spectrum resources (CUEs) with cellular users in a time-splitting manner. Besides, D2D nodes could reuse time slots preoccupied by CUEs according to a location-based spectrum access (LSA) strategy on the premise of cellular communication quality. The key challenge lies in that D2D nodes have no information on the LSA strategy and the access principle of CUEs. Thus, we design a DRL-based spectrum access scheme such that the D2D nodes can autonomously acquire an optimal strategy for efficient spectrum access without any prior knowledge to achieve a specific objective such as maximizing the normalized sum throughput. Moreover, we adopt a generalized double deep $Q$ -network (DDQN) algorithm and extend the objective function to explore the resource allocation fairness for D2D nodes. The proposed scheme is evaluated under various conditions and our simulation results show that it can achieve the near-optimal throughput performance with different objectives compared to the benchmark, which is the theoretical throughput upper bound derived from a genius-aided scheme with complete system knowledge available.
Over the past two decades, Long Short-Term Memory (LSTM) networks have been used to solve problems that require modeling of long sequence because they can selectively remember certain patterns over a long period, thus outperforming traditional feed-forward neural networks and Recurrent Neural Network (RNN) on learning long-term dependencies. However, LSTM is characterized by feedback dependence, which limits the high parallelism of general-purpose processors such as CPU and GPU. Besides, in terms of the energy efficiency of data center applications, the high consumption of GPU and CPU computing cannot be ignored. To deal with the above problems, Field Programmable Gate Array (FPGA) is becoming an ideal alternative. FPGA has the characteristics of low power consumption and low latency, which are helpful for the acceleration and optimization of LSTM and other RNNs. This paper proposes an implementation scheme of the LSTM network acceleration engine based on FPGA and further optimizes the implementation through fixed-point arithmetic, systolic array and lookup table for nonlinear function. On this basis, for easy deployment and application, we integrate the proposed acceleration engine into Caffe, one of the most popular deep learning frameworks. Experimental results show that, compared with CPU and GPU, the FPGA-based acceleration engine can achieve performance improvement of 8.8 and 2.2 times and energy efficiency improvement of 16.9 and 9.6 times, respectively, within Caffe framework.