This paper presents a comprehensive literature review on applications of deep reinforcement learning in communications and networking. Modern networks, e.g., Internet of Things (IoT) and Unmanned Aerial Vehicle (UAV) networks, become more decentralized and autonomous. In such networks, network entities need to make decisions locally to maximize the network performance under uncertainty of network environment. Reinforcement learning has been efficiently used to enable the network entities to obtain the optimal policy including, e.g., decisions or actions, given their states when the state and action spaces are small. However, in complex and large-scale networks, the state and action spaces are usually large, and the reinforcement learning may not be able to find the optimal policy in reasonable time. Therefore, deep reinforcement learning, a combination of reinforcement learning with deep learning, has been developed to overcome the shortcomings. In this survey, we first give a tutorial of deep reinforcement learning from fundamental concepts to advanced models. Then, we review deep reinforcement learning approaches proposed to address emerging issues in communications and networking. The issues include dynamic network access, data rate control, wireless caching, data offloading, network security, and connectivity preservation which are all important to next generation networks such as 5G and beyond. Furthermore, we present applications of deep reinforcement learning for traffic routing, resource sharing, and data collection. Finally, we highlight important challenges, open issues, and future research directions of applying deep reinforcement learning.

This paper presents a literature review on recent applications and design aspects of the intelligent reflecting surface (IRS) in the future wireless networks. Conventionally, the network optimization has been limited to transmission control at two endpoints, i.e., end users and network controller. The fading wireless channel is uncontrollable and becomes one of the main limiting factors for performance improvement. The IRS is composed of a large array of scattering elements, which can be individually configured to generate additional phase shifts to the signal reflections. Hence, it can actively control the signal propagation properties in favor of signal reception, and thus realize the notion of a smart radio environment. As such, the IRS’s phase control, combined with the conventional transmission control, can potentially bring performance gain compared to wireless networks without IRS. In this survey, we first introduce basic concepts of the IRS and the realizations of its reconfigurability. Then, we focus on applications of the IRS in wireless communications. We overview different performance metrics and analytical approaches to characterize the performance improvement of IRS-assisted wireless networks. To exploit the performance gain, we discuss the joint optimization of the IRS’s phase control and the transceivers’ transmission control in different network design problems, e.g., rate maximization and power minimization problems. Furthermore, we extend the discussion of IRS-assisted wireless networks to some emerging use cases. Finally, we highlight important practical challenges and future research directions for realizing IRS-assisted wireless networks in beyond 5G communications.

Recently, ambient backscatter communications has been introduced as a cutting-edge technology which enables smart devices to communicate by utilizing ambient radio frequency (RF) signals without requiring active RF transmission. This technology is especially effective in addressing communication and energy efficiency problems for low-power communications systems such as sensor networks. It is expected to realize numerous Internet-of-Things (IoT) applications. Therefore, this paper aims to provide a contemporary and comprehensive literature review on fundamentals, applications, challenges, and research efforts/progress of ambient backscatter communications. In particular, we first present fundamentals of backscatter communications and briefly review bistatic backscatter communications systems. Then, the general architecture, advantages, and solutions to address existing issues and limitations of ambient backscatter communications systems are discussed. Additionally, emerging applications of ambient backscatter communications are highlighted. Finally, we outline some open issues and future research directions.

Radio waves carry both energy and information simultaneously. Nevertheless, Radio-Frequency (RF) transmissions of these quantities have traditionally been treated separately. Currently, the community is experiencing a paradigm shift in wireless network design, namely unifying wireless transmission of information and power so as to make the best use of the RF spectrum and radiation as well as the network infrastructure for the dual purpose of communicating and energizing. In this paper, we review and discuss recent progress in laying the foundations of the envisioned dual purpose networks by establishing a signal theory and design for Wireless Information and Power Transmission (WIPT) and identifying the fundamental tradeoff between conveying information and power wirelessly. We start with an overview of WIPT challenges and technologies, namely Simultaneous Wireless Information and Power Transfer (SWIPT), Wirelessly Powered Communication Networks (WPCNs), and Wirelessly Powered Backscatter Communication (WPBC). We then characterize energy harvesters and show how WIPT signal and system designs crucially revolve around the underlying energy harvester model. To that end, we highlight three different energy harvester models, namely one linear model and two nonlinear models, and show how WIPT designs differ for each of them in single-user and multi-user deployments. Topics discussed include rate-energy region characterization, transmitter and receiver architectures, waveform design, modulation, beamforming and input distribution optimizations, resource allocation, and RF spectrum use. We discuss and check the validity of the different energy harvester models and the resulting signal theory and design based on circuit simulations, prototyping and experimentation. We also point out numerous directions that are promising for future research.

In the Internet of Vehicles (IoV), data sharing among vehicles is critical to improving driving safety and enhancing vehicular services. To ensure security and traceability of data sharing, existing studies utilize efficient Delegated Proof-of-Stake consensus scheme as hard security solutions to establish blockchainenabled IoV (BIoV). However, as the miners are selected from miner candidates by stake-based voting, defending against voting collusion between the candidates and compromised high-stake vehicles becomes challenging. To address the challenge, in this paper, we propose a two-stage soft security enhancement solution: (i) miner selection and (ii) block verification. In the first stage, we design a reputation-based voting scheme to ensure secure miner selection. This scheme evaluates candidates' reputation using both past interactions and recommended opinions from other vehicles. The candidates with high reputation are selected to be active miners and standby miners. In the second stage, to prevent internal collusion among active miners, a newly generated block is further verified and audited by standby miners. To incentivize the participation of the standby miners in block verification, we adopt the contract theory to model the interactions between active miners and standby miners, where block verification security and delay are taken into consideration. Numerical results based on a real-world dataset confirm the security and efficiency of our schemes for data sharing in BIoV.