This paper investigates the resource allocation problem in vehicular communications based on multi-agent Deep Deterministic Policy Gradient (DDPG), in which each Vehicle-to-Vehicle (V2V) communication acts as agent and adopts Non-Orthogonal Multiple Access (NOMA) technology to share the frequency spectrum that pre-allocated to Vehicle-to-Infrastructure (V2I) communications. Different with conventional D2D communications, the fast varying channel condition due to the high mobility in vehicular environment causes the difficulty of collecting instantaneous Channel State Information (CSI) at base station. Meanwhile, one tremendous challenge faced by vehicular communications is how to maximize the sum-rate of V2I communications simultaneously guaranteeing the latency and reliability requirements for the transmission of safety-critical information in V2V communications. In response, we formulate the resource allocation problem as a decentralized Discrete-time and Finite-state Markov Decision Process (DFMDP), in which allocation decisions are made by multiple agents that do not have complete and global network information. Due to the complexity of the problem, we propose a DDPG algorithm which is capable of handling continuous high dimensional action spaces to find the optimal allocation strategy. Numerical results verify that each agent can effectively learn from the environment by means of the proposed DDPG algorithm to maximize the sum-rate of V2I communications while satisfying the stringent latency and reliability constraints of V2V communications.

In this paper, we investigate the joint design of beamforming vectors and trajectory for unmanned aerial vehicles (UAVs) in integrated sensing and communications networks, aiming to maximize the achievable covert rate (ACR) for legitimate users against multiple passive wardens. Considering the worst-case scenario, where the wardens strategically select optimal decision thresholds, we derive the minimum detection error probability and incorporate covertness constraints within the beamforming scheme. Our approach entails formulating the design as a non-convex optimization problem for maximizing the average ACR along the UAV trajectory. The formulation takes into account various practical constraints, such as the maximum transmit power, UAV flight speed limitations, minimum beamforming gain towards sensing targets, and the detection probability threshold for wardens. To address this intricate problem, we propose a block coordinate descent-based optimization algorithm. This algorithm alternates between updating beamforming vectors and UAV trajectories, offering a high-quality suboptimal solution to the original problem. Theoretical analyses reveal that when the detection probability threshold is sufficiently small, a linear correlation emerges between the maximum relative variation ratio in the average received signal power at wardens under two hypotheses and the detection probability. Furthermore, to enhance covertness against the wardens, it is necessary to either decrease the projection of information beamforming covariance matrix or increase the projection of sensing beamforming covariance matrix onto the subspace spanned by the eavesdropping channel vectors. Finally, extensive simulations are presented to validate the covert performance enhancements of our proposed methodology, compared with various baseline schemes adopting existing approaches.

Wireless body area networks (WBANs) have attracted great attention from both industry and academia as a promising technology for continuous monitoring of physiological signals of the human body. As the sensors in WBANs are typically battery-driven and inconvenient to recharge, an energy efficient resource allocation scheme is essential to prolong the lifetime of the networks, while guaranteeing the rigid requirements of quality of service (QoS) of the WBANs in nature. As a possible alternative solution to address the energy efficiency problem, energy harvesting (EH) technology with the capability of harvesting energy from ambient sources can potentially reduce the dependence on the battery supply. Consequently, in this paper, we investigate the resource allocation problem for EH-powered WBANs (EH-WBANs). Our goal is to maximize the energy efficiency of the EH-WBANs with the joint consideration of transmission mode, relay selection, allocated time slot, transmission power, and the energy constraint of each sensor. In view of the characteristic of the EH-WBANs, we formulate the energy efficiency problem as a discrete-time and finite-state Markov decision process (DFMDP), in which allocation strategy decisions are made by a hub that does not have complete and global network information. Owing to the complexity of the problem, we propose a modified Q-learning (QL) algorithm to obtain the optimal allocation strategy. The numerical results validate the effectiveness of the proposed scheme as well as the low computation complexity of the proposed modified Q-learning (QL) algorithm.

In this article, we propose a resource allocation scheme for air slicing in unmanned aerial vehicle (UAV)-assisted cellular vehicle-to-everything (V2X) communications. We consider a scenario, where multiple flexible UAVs are deployed as the aerial base station (BS) to assist terrestrial BS for providing service to vehicular users with the objective of maximizing the bandwidth efficiency while concurrently guaranteeing the transmission rate and the latency by adopting network slicing. Due to the uncertainty of the stochastic environment in the scenario, we formulate the optimization problem to be a stochastic game, which is an extension of game theory to Markov decision process-like environments for the case of multiple adaptive agents are involved to compete goals simultaneously. Nevertheless, the dynamic nature of both UAVs and vehicles pose the difficulty of perceiving and interacting with the unknown environment, the long short-term memory algorithm is used for extracting the features of the observation and making forecast on the mobility of UAVs and vehicles. Simulation results adduce the validity of the proposed scheme as compared with two benchmark schemes: Deep Q-network and deep deterministic policy gradient.

By providing spatial diversity gain, the incorporation of multiple antennas into mobile edge computing (MEC) systems can improve the transmission performance. Meanwhile, employing energy harvesting (EH) helps enhance the system sustainability. In this paper, we focus on multi-input multi-output (MIMO) MEC systems with EH and studies the computation offloading. The design objective is to minimize the time average of a weighted sum of energy consumption and execution delay, meanwhile stabilizing the battery energy queue. To this end, we formulate the problem as a statistic program and propose a dynamic computation offloading (DCO) algorithm in which the transmitter covariance matrix, CPU-cycle frequencies for local computing, and partial offloading ratio are jointly optimized. Based on Lyapunov optimization, the program is first transformed into a nonconvex per-time slot problem. Then, we solve it by the successive convex approximation (SCA) technique, where a sequence of convex problems are created and solved. Simulation results demonstrate that the proposed algorithm is asymptotically optimal and outperforms several benchmark schemes in terms of both the average system cost and task drop ratio.

This letter investigates the resource allocation problem for multiple Unmanned Aerial Vehicles (UAVs)-served Machine-to-Machine (M2M) communications. Our goal is to maximize the sum-rate of UAVs-served M2M communications by jointly considering the transmission power, transmission mode, frequency spectrum, relay selection and the trajectory of UAVs. In order to model the uncertainty of stochastic environments, we formulate the resource allocation problem to be a Markov game, which is the generalization of Markov Decision Process (MDP) for the case of multiple agents. However, owning to the UAVs mobility poses the difficulty of perceiving the environment, we propose a Long Short-Term Memory (LSTM) with Generative Adversarial Networks (GANs) framework to better track and forecast the UAVs mobility and improving the network reward. Numerical results demonstrate that the proposed framework outperforms the conventional LSTM and Deep Q-Network (DQN) algorithms.