In this paper, we consider delay-optimal charging scheduling of the electric vehicles (EVs) at a charging station with multiple charge points. The charging station is equipped with renewable energy generation devices and can also buy energy from power grids. The uncertainty of the arrival of the EV, the intermittence of the renewable energy, and the variation of the grid power price are taken into account and described as independent Markov processes. Meanwhile, the required charging energy for each EV is random. The goal is to minimize the mean waiting time for EVs under the long-term constraint on the cost. We propose queue mapping to convert the EV queue to the charging demand queue, and we prove the equivalence between the minimization of the two queues' average length. Then, we focus on the minimization for the average length of the charging demand queue under the long-term cost constraint. We propose a framework of Markov decision process (MDP) to investigate this constrained stochastic optimization problem. The system state includes the charging demand queue length, the charging demand arrival, the energy level in the storage battery of the renewable energy, the renewable energy arrival, and the grid power price. Additionally, the number of charging demands and the allocated energy from the storage battery compose the 2-D policy. We derive two necessary conditions of the optimal policy. Moreover, we discuss the reduction of the 2-D policy to be the number of charging demands only. We give the sets of system states for which charging no demand and charging as many demands as possible are optimal, respectively. Finally, we investigate the proposed policies numerically.

Over-the-air computation (AirComp) based federated learning (FL) is capable of achieving fast model aggregation by exploiting the waveform superposition property of multiple-access channels. However, the model aggregation performance is severely limited by the unfavorable wireless propagation channels. In this paper, we propose to leverage intelligent reflecting surface (IRS) to achieve fast yet reliable model aggregation for AirComp-based FL. To optimize the learning performance, we present the convergence analysis of our proposed IRS-assisted AirComp-based FL system, based on which we propose to maximize the number of scheduled devices of each communication round under certain mean-squared error (MSE) requirements. To tackle the formulated highly-intractable problem, we propose a two-step optimization framework. Specifically, we induce the sparsity of device selection in the first step, followed by solving a series of MSE minimization problems to find the maximum feasible device set in the second step. We then propose an alternating optimization framework, supported by the difference-of-convex programming for low-rank optimization, to efficiently design the aggregation beamformers at the BS and phase shifts at the IRS. Simulation results demonstrate that our proposed algorithm and the deployment of an IRS can achieve a higher FL prediction accuracy than the baseline schemes.

Internet of Things (IoT) technology suffers from the challenge that rare wireless network resources are difficult to meet the influx of a huge number of terminal devices. Cache-enabled Device-to-Device (D2D) communication technology is expected to relieve network pressure with the facts that the requesting contents can be easily obtained from nearby users. However, how to design an effective caching policy becomes very challenging due to the limited content storage capacity and the uncertainty of user mobility pattern. In this paper, we study the jointly cache content placement and delivery policy for the cache-enabled D2D networks. Specifically, two potential recurrent neural network approaches (the echo state network (ESN) and long-short term memory (LSTM) network) are employed to predict users’ mobility and content popularity, so as to determine which content to cache and where to cache. When the local cache of the user cannot satisfy its own request, the user may consider establishing a D2D link with the neighboring user to implement the content delivery. In order to decide which user will be selected to establish the D2D link, we propose the novel schemes based on deep reinforcement learning to implement the dynamic decision-making and optimization of the content delivery problems, aiming at improving the quality of experience of overall caching system. The simulation results suggest that the cache hit ratio of the system can be well improved by the proposed content placement strategy, and the proposed content delivery approaches can effectively reduce the request content delivery delay and energy consumption.

Over-the-air computation (AirComp) is a disruptive technique for fast wireless data aggregation in Internet of Things (IoT) networks via exploiting the waveform superposition property of multiple-access channels. However, the performance of AirComp is bottlenecked by the worst channel condition among all links between the IoT devices and the access point. In this paper, a reconfigurable intelligent surface (RIS) assisted AirComp system is proposed to boost the received signal power and thus mitigate the performance bottleneck by reconfiguring the propagation channels. With an objective to minimize the AirComp distortion, we propose a joint design of AirComp transceivers and RIS phase-shifts, which however turns out to be a highly intractable non-convex programming problem. To this end, we develop a novel alternating minimization framework in conjunction with the successive convex approximation technique, which is proved to converge monotonically. To reduce the computational complexity, we transform the subproblem in each alternation as a smooth convex-concave saddle point problem, which is then tackled by proposing a Mirror-Prox method that only involves a sequence of closed-form updates. Simulations show that the computation time of the proposed algorithm can be two orders of magnitude smaller than that of the state-of-the-art algorithms, while achieving a similar distortion performance.

This article investigates simultaneous transmitting and reflecting reconfigurable intelligent surface (STAR-RIS)-assisted downlink multiuser multiple-input–single-output (MU-MISO) networks with the rate splitting multiple access (RSMA) scheme. A base station (BS) desires to simultaneously transmit messages to multiple users with the assistance of an STAR-RIS to enhance communication quality as well as extend the coverage of users. An optimization problem is formulated to maximize the achievable sum rate of the networks on the premise of satisfying the constraints on power budget at the BS, total common-stream rate of users, and individual users’ minimum rate requirements, via jointly optimizing the beamforming vectors, the common-stream rate allocation vector, and the transmission and reflection coefficients (TARCs) matrix. Due to the dynamic changes of communication links and the coupling of multiple variables, it is challenging to solve such a nonconvex optimization problem by utilizing traditional methods. Therefore, a proximal policy optimization (PPO)-based deep reinforcement learning (DRL) algorithm is proposed, where the reward function, the action space and the state space are designed properly. A constraint-satisfaction-processing (CSP) method is employed to further adjust the optimized transmit power to make sure that the obtained optimized results satisfy the power budget constraint. Simulation results show that the proposed PPO-based DRL algorithm converges well and achieves much better performance than several baselines, such as the soft actor–critic (SAC), the deep deterministic policy gradient (DDPG), the genetic algorithm (GA), the maximum ratio transmission (MRT), the zero-forcing (ZF), and the random methods. Moreover, it demonstrates that deploying STAR-RIS greatly enhances the system sum rate and user fairness compared to deploying traditional reflecting-only RIS (RO-RIS) and without RIS. Besides, it also shows that adopting the RSMA scheme achieves more notable performance gains than the nonorthogonal multiple access (NOMA) scheme in such a network.

Attracted by the advantages of multi-access edge computing (MEC) and non-orthogonal multiple access (NOMA), this paper studies the resource allocation problem of a NOMA-MEC system in an ultra-dense network (UDN), where each user may opt for offloading tasks to the MEC server when it is computationally intensive. Our optimization goal is to minimize the system computation cost, concerning the energy consumption and task delay of users. In order to tackle the non-convexity issue of the objective function, we decouple this problem into two sub-problems: user clustering as well as jointly power and computation resource allocation. Firstly, we propose a user clustering matching (UCM) algorithm exploiting the differences in channel gains of users. Then, relying on the mean-field game (MFG) framework, we solve the resource allocation problem for intensive user deployment, using the novel deep deterministic policy gradient (DDPG) method, which is termed by a mean-field-deep deterministic policy gradient (MF-DDPG) algorithm. Finally, a jointly iterative optimization algorithm (JIOA) of UCM and MF-DDPG is proposed to minimize the computation cost of users. The simulation results demonstrate that the proposed algorithm exhibits rapid convergence, and is capable of efficiently reducing both the energy consumption and task delay of users.