Internet of Things (IoT) technology suffers from the challenge that rare wireless network resources are difficult to meet the influx of a huge number of terminal devices. Cache-enabled Device-to-Device (D2D) communication technology is expected to relieve network pressure with the facts that the requesting contents can be easily obtained from nearby users. However, how to design an effective caching policy becomes very challenging due to the limited content storage capacity and the uncertainty of user mobility pattern. In this paper, we study the jointly cache content placement and delivery policy for the cache-enabled D2D networks. Specifically, two potential recurrent neural network approaches (the echo state network (ESN) and long-short term memory (LSTM) network) are employed to predict users’ mobility and content popularity, so as to determine which content to cache and where to cache. When the local cache of the user cannot satisfy its own request, the user may consider establishing a D2D link with the neighboring user to implement the content delivery. In order to decide which user will be selected to establish the D2D link, we propose the novel schemes based on deep reinforcement learning to implement the dynamic decision-making and optimization of the content delivery problems, aiming at improving the quality of experience of overall caching system. The simulation results suggest that the cache hit ratio of the system can be well improved by the proposed content placement strategy, and the proposed content delivery approaches can effectively reduce the request content delivery delay and energy consumption.

Attracted by the advantages of multi-access edge computing (MEC) and non-orthogonal multiple access (NOMA), this paper studies the resource allocation problem of a NOMA-MEC system in an ultra-dense network (UDN), where each user may opt for offloading tasks to the MEC server when it is computationally intensive. Our optimization goal is to minimize the system computation cost, concerning the energy consumption and task delay of users. In order to tackle the non-convexity issue of the objective function, we decouple this problem into two sub-problems: user clustering as well as jointly power and computation resource allocation. Firstly, we propose a user clustering matching (UCM) algorithm exploiting the differences in channel gains of users. Then, relying on the mean-field game (MFG) framework, we solve the resource allocation problem for intensive user deployment, using the novel deep deterministic policy gradient (DDPG) method, which is termed by a mean-field-deep deterministic policy gradient (MF-DDPG) algorithm. Finally, a jointly iterative optimization algorithm (JIOA) of UCM and MF-DDPG is proposed to minimize the computation cost of users. The simulation results demonstrate that the proposed algorithm exhibits rapid convergence, and is capable of efficiently reducing both the energy consumption and task delay of users.

With the rapid development of deep learning (DL) in recent years, automatic modulation recognition (AMR) with DL has achieved high accuracy. However, aiming to obtain higher classification accuracy, DL requires numerous training samples. In order to solve this problem, it is a challenge to study how to efficiently use DL for AMR in the case of few samples. In this letter, inspired by the capsule network (CapsNet), we propose a new network structure named AMR-CapsNet to achieve higher classification accuracy of modulation signals with fewer samples, and further analyze the adaptability of DL models in the case of few samples. The simulation results demonstrate that when 3% of the dataset is used to train and the signal-to-noise ratio (SNR) is greater than 2dB, the overall classification accuracy of the AMR-CapsNet is greater than 80%. Compared with convolutional neural network (CNN), the classification accuracy is improved by 20%.

The Internet of Things (IoT) technology has recently drawn more attention due to its ability to achieve the interconnections of massive physic devices. However, how to provide a reliable power supply to energy-constrained devices and improve the energy efficiency in the wireless powered IoT (WP-IoT) is a twofold challenge. In this paper, we develop a novel wireless power transmission (WPT) system, where an unmanned aerial vehicle (UAV) equipped with radio frequency energy transmitter charges the IoT devices. A machine learning framework of echo state networks (ESNs) together with an improved k-means clustering algorithm is used to predict the energy consumption and cluster all the sensor nodes at the next period, thus automatically determining the charging strategy. The energy obtained from the UAV by WPT supports the IoT devices to communicate with each other. In order to improve the energy efficiency of WP-IoT system, the interference mitigation problem is modeled as a mean field game (MFG), where an optimal power control policy is presented to adapt and analyze the large number of sensor nodes randomly deployed in WP-IoT. The numerical results verify that our proposed dynamic charging policy effectively reduce the data packet loss rate, and that the optimal power control policy greatly mitigate the interference, and improve the energy efficiency of the whole network.

As an emerging technology in 5G, ultra-dense unmanned aerial vehicles (UAVs) network can significantly improve the system capacity and networks coverage. However, it is still a challenge to reduce interference and improve energy efficiency (EE) of UAVs. In this paper, we investigate a downlink power control problem to maximize the EE in an ultra-dense UAV network. Firstly, the power control problem is formulated as a discrete mean field game (MFG) to imitate the interactions among a large number of UAVs, and then the MFG framework is transformed into a Markov decision process (MDP) to obtain the equilibrium solution of the MFG due to the dense deployment of UAVs. Specifically, a deep reinforcement learning-based MFG (DRL-MFG) algorithm is proposed to suppress the interference and maximize the EE by using deep neural networks (DNN) to explore the optimal power strategy for UAVs. The numerical results show that the UAVs can effectively interact with the environment to obtain the optimal power control strategy. Compared with the benchmarks algorithms, the DRL-MFG algorithm converges faster to the solution of MFG and improves the EE of UAVs. Moreover, the impact of the transmit power on EE under the different heights of the UAVs is also analyzed.

In unmanned aerial vehicle (UAV)-assisted massive multi-input multi-output (MIMO) millimeter-wave (mmWave) networks, beam-steering guarantees reliable and steady connection between flying base stations and ground users with the challenge of strict angular deviation. In this paper, we investigate a joint optimization problem of beamforming and beam-steering in the multi-UAV mmWave networks, considering line-of-sight (LoS) communication for UAVs. For the hybrid beamforming optimization of massive MIMO mmWave, we propose a hybrid beamforming scheme based on the cross-entropy estimation with the robustness algorithm inspired by machine learning, which aims to optimize the hybrid precoding matrix. For the beamsteering optimization, we propose a novel mean field game (MFG)-based massive MIMO angle control scheme to model the optimal mmWave channel optimization problem between UAVs and ground users. In addition, when dealing with the problem of initial sensitivity and difficulty to solve the partial differential equations in the MFG, we utilize reinforcement learning to achieve the mean field equilibrium, which is described as the mean field learning game algorithm. Finally, a joint beamforming and beam-steering optimization algorithm is proposed to maximize the system sum-rate. Simulation results show the significant improvements in sum-rate, energy efficiency, and spectral efficiency, which verify the effectiveness of the proposed algorithm.