This paper investigates the resource allocation problem in vehicular communications based on multi-agent Deep Deterministic Policy Gradient (DDPG), in which each Vehicle-to-Vehicle (V2V) communication acts as agent and adopts Non-Orthogonal Multiple Access (NOMA) technology to share the frequency spectrum that pre-allocated to Vehicle-to-Infrastructure (V2I) communications. Different with conventional D2D communications, the fast varying channel condition due to the high mobility in vehicular environment causes the difficulty of collecting instantaneous Channel State Information (CSI) at base station. Meanwhile, one tremendous challenge faced by vehicular communications is how to maximize the sum-rate of V2I communications simultaneously guaranteeing the latency and reliability requirements for the transmission of safety-critical information in V2V communications. In response, we formulate the resource allocation problem as a decentralized Discrete-time and Finite-state Markov Decision Process (DFMDP), in which allocation decisions are made by multiple agents that do not have complete and global network information. Due to the complexity of the problem, we propose a DDPG algorithm which is capable of handling continuous high dimensional action spaces to find the optimal allocation strategy. Numerical results verify that each agent can effectively learn from the environment by means of the proposed DDPG algorithm to maximize the sum-rate of V2I communications while satisfying the stringent latency and reliability constraints of V2V communications.

Wireless body area networks (WBANs) have attracted great attention from both industry and academia as a promising technology for continuous monitoring of physiological signals of the human body. As the sensors in WBANs are typically battery-driven and inconvenient to recharge, an energy efficient resource allocation scheme is essential to prolong the lifetime of the networks, while guaranteeing the rigid requirements of quality of service (QoS) of the WBANs in nature. As a possible alternative solution to address the energy efficiency problem, energy harvesting (EH) technology with the capability of harvesting energy from ambient sources can potentially reduce the dependence on the battery supply. Consequently, in this paper, we investigate the resource allocation problem for EH-powered WBANs (EH-WBANs). Our goal is to maximize the energy efficiency of the EH-WBANs with the joint consideration of transmission mode, relay selection, allocated time slot, transmission power, and the energy constraint of each sensor. In view of the characteristic of the EH-WBANs, we formulate the energy efficiency problem as a discrete-time and finite-state Markov decision process (DFMDP), in which allocation strategy decisions are made by a hub that does not have complete and global network information. Owing to the complexity of the problem, we propose a modified Q-learning (QL) algorithm to obtain the optimal allocation strategy. The numerical results validate the effectiveness of the proposed scheme as well as the low computation complexity of the proposed modified Q-learning (QL) algorithm.

In this article, we propose a resource allocation scheme for air slicing in unmanned aerial vehicle (UAV)-assisted cellular vehicle-to-everything (V2X) communications. We consider a scenario, where multiple flexible UAVs are deployed as the aerial base station (BS) to assist terrestrial BS for providing service to vehicular users with the objective of maximizing the bandwidth efficiency while concurrently guaranteeing the transmission rate and the latency by adopting network slicing. Due to the uncertainty of the stochastic environment in the scenario, we formulate the optimization problem to be a stochastic game, which is an extension of game theory to Markov decision process-like environments for the case of multiple adaptive agents are involved to compete goals simultaneously. Nevertheless, the dynamic nature of both UAVs and vehicles pose the difficulty of perceiving and interacting with the unknown environment, the long short-term memory algorithm is used for extracting the features of the observation and making forecast on the mobility of UAVs and vehicles. Simulation results adduce the validity of the proposed scheme as compared with two benchmark schemes: Deep Q-network and deep deterministic policy gradient.

This letter investigates the resource allocation problem for multiple Unmanned Aerial Vehicles (UAVs)-served Machine-to-Machine (M2M) communications. Our goal is to maximize the sum-rate of UAVs-served M2M communications by jointly considering the transmission power, transmission mode, frequency spectrum, relay selection and the trajectory of UAVs. In order to model the uncertainty of stochastic environments, we formulate the resource allocation problem to be a Markov game, which is the generalization of Markov Decision Process (MDP) for the case of multiple agents. However, owning to the UAVs mobility poses the difficulty of perceiving the environment, we propose a Long Short-Term Memory (LSTM) with Generative Adversarial Networks (GANs) framework to better track and forecast the UAVs mobility and improving the network reward. Numerical results demonstrate that the proposed framework outperforms the conventional LSTM and Deep Q-Network (DQN) algorithms.

By offloading some computational tasks to the edge server, edge computing can help relieve the increasing computation burden of mobile users and improve the quality of experience of users. In this article, we investigate the computation offloading in edge computing-enabled multiuser cell-free (CF) multi-input multioutput (MIMO) networks with multiple building baseband units (BBUs). The user association, transmit covariances, CPU-cycle frequencies, and allocated bandwidths are jointly optimized to minimize two objectives: 1) the energy consumption of mobile devices (MDs) and 2) execution delay of tasks. We formulate it as a vector optimization problem and adopt the scalarization technique to transform it into a scalar mixed-integer nonlinear programming (MINLP). Then, to solve the MINLP, we introduce user sorting into the framework of branch and bound and propose a sorted MD association (MDA) method. Two key issues are tackled in the proposed MDA, i.e., the lower bound of the MINLP and the updation of incumbent solution. For the first issue, we present a Lagrangian dual relaxation algorithm based on subgradient projection, while the second one involving another nonconvex minimization problem, we propose a successive convex approximation (SCA)-based algorithm to solve it. Finally, extensive simulations demonstrate that the proposed method is able to reduce the system cost significantly and achieve better system performance by comparing with the benchmark schemes.