This paper presents a comprehensive literature review on applications of deep reinforcement learning in communications and networking. Modern networks, e.g., Internet of Things (IoT) and Unmanned Aerial Vehicle (UAV) networks, become more decentralized and autonomous. In such networks, network entities need to make decisions locally to maximize the network performance under uncertainty of network environment. Reinforcement learning has been efficiently used to enable the network entities to obtain the optimal policy including, e.g., decisions or actions, given their states when the state and action spaces are small. However, in complex and large-scale networks, the state and action spaces are usually large, and the reinforcement learning may not be able to find the optimal policy in reasonable time. Therefore, deep reinforcement learning, a combination of reinforcement learning with deep learning, has been developed to overcome the shortcomings. In this survey, we first give a tutorial of deep reinforcement learning from fundamental concepts to advanced models. Then, we review deep reinforcement learning approaches proposed to address emerging issues in communications and networking. The issues include dynamic network access, data rate control, wireless caching, data offloading, network security, and connectivity preservation which are all important to next generation networks such as 5G and beyond. Furthermore, we present applications of deep reinforcement learning for traffic routing, resource sharing, and data collection. Finally, we highlight important challenges, open issues, and future research directions of applying deep reinforcement learning.

This paper presents a literature review on recent applications and design aspects of the intelligent reflecting surface (IRS) in the future wireless networks. Conventionally, the network optimization has been limited to transmission control at two endpoints, i.e., end users and network controller. The fading wireless channel is uncontrollable and becomes one of the main limiting factors for performance improvement. The IRS is composed of a large array of scattering elements, which can be individually configured to generate additional phase shifts to the signal reflections. Hence, it can actively control the signal propagation properties in favor of signal reception, and thus realize the notion of a smart radio environment. As such, the IRS’s phase control, combined with the conventional transmission control, can potentially bring performance gain compared to wireless networks without IRS. In this survey, we first introduce basic concepts of the IRS and the realizations of its reconfigurability. Then, we focus on applications of the IRS in wireless communications. We overview different performance metrics and analytical approaches to characterize the performance improvement of IRS-assisted wireless networks. To exploit the performance gain, we discuss the joint optimization of the IRS’s phase control and the transceivers’ transmission control in different network design problems, e.g., rate maximization and power minimization problems. Furthermore, we extend the discussion of IRS-assisted wireless networks to some emerging use cases. Finally, we highlight important practical challenges and future research directions for realizing IRS-assisted wireless networks in beyond 5G communications.

Mobile edge computing (MEC) is an emerging technology to support resource-intensive yet delay-sensitive applications using small cloud-computing platforms deployed at the mobile network edges. However, the existing MEC techniques are not applicable to the situation where the number of mobile users increases explosively or the network facilities are sparely distributed. In view of this insufficiency, unmanned aerial vehicles (UAVs) have been employed to improve the connectivity of ground Internet-of-Things (IoT) devices due to their high altitude. This paper proposes an innovative UAV-enabled MEC system involving the interactions among IoT devices, UAV and edge clouds (ECs). The system deploys and operates a UAV properly to facilitate the MEC service provisioning to a set of IoT devices in regions where existing ECs cannot be accessible to IoT devices due to terrestrial signal blockage or shadowing. The UAV and ECs in the system collaboratively provide MEC services to the IoT devices. For optimal service provisioning in this system, we formulate an optimization problem aiming at minimizing the weighted sum of the service delay of all IoT devices and UAV energy consumption by jointly optimizing UAV position, communication and computing resource allocation, and task splitting decisions. However, the resulting optimization problem is highly non-convex and thus difficult to solve optimally. To tackle this problem, we develop an efficient algorithm based on successive convex approximation to obtain sub-optimal solutions. Numerical experiments demonstrate that our proposed collaborative UAV-EC offloading scheme largely outperforms baseline schemes that solely rely on UAV or edge clouds for MEC in IoT.

This paper proposes to use Intelligent Reflecting Surface (IRS) as a green jammer to attack a legitimate communication without using any internal energy to generate jamming signals. In particular, the IRS is used to intelligently reflect the signals from the legitimate transmitter to the legitimate receiver (LR) to guarantee that the received signals from direct and reflecting links can be added destructively, which thus diminishes the Signal-to-Interference-plus-Noise Ratio (SINR) at the LR. To minimize the received signal power at the LR, we consider the joint optimization of magnitudes of reflection coefficients and discrete phase shifts at the IRS. Based on the block coordinate descent, semidefinite relaxation, and Gaussian randomization techniques, the solution can be obtained efficiently. Through simulation results, we show that by using the IRS-based jammer, we can reduce the signal power received at the LR by up to 99%. Interestingly, the performance of the proposed IRS-based jammer is even better than that of the conventional active jamming attacks in some scenarios.

The integration of wireless power transfer (WPT) with the low-power backscatter communications provides a promising way to sustain battery-less wireless networks. In this paper, we consider a backscatter communication network wirelessly powered by a power beacon station (PBS). Each backscatter radio uses the harvested energy to power its data transmissions, in which some other radios can help as the wireless relays with an aim to improve throughput performance by cooperative transmission. Under this setting, we formulate a throughput maximization problem to jointly optimize WPT and the relay strategy of the backscatter radios. An iterative algorithm with reduced complexity and communication overhead is proposed to decompose the original problem into two sub-problems distributed at the PBS and the backscatter receiver. Moreover, we take uncertain channel information into consideration and formulate robust counter-parts of the throughput maximization problem when either the backscatter or relay channel is subject to estimation errors. The difficulty of the robust counter-part lies in the coupling of the PBS' power allocation and relay strategy in matrix inequalities, which is addressed by alternating optimization with guaranteed convergence. Numerical results reveal that the cooperative relay strategy of the backscatter radios significantly improves the throughput performance.