Reinforcement learning (RL) algorithms have been around for decades and employed to solve various sequential decision-making problems. These algorithms, however, have faced great challenges when dealing with high-dimensional environments. The recent development of deep learning has enabled RL methods to drive optimal policies for sophisticated and capable agents, which can perform efficiently in these challenging environments. This article addresses an important aspect of deep RL related to situations that require multiple agents to communicate and cooperate to solve complex tasks. A survey of different approaches to problems related to multiagent deep RL (MADRL) is presented, including nonstationarity, partial observability, continuous state and action spaces, multiagent training schemes, and multiagent transfer learning. The merits and demerits of the reviewed methods will be analyzed and discussed with their corresponding applications explored. It is envisaged that this review provides insights about various MADRL methods and can lead to the future development of more robust and highly useful multiagent learning methods for solving real-world problems.

The scale of Internet-connected systems has increased considerably, and these systems are being exposed to cyberattacks more than ever. The complexity and dynamics of cyberattacks require protecting mechanisms to be responsive, adaptive, and scalable. Machine learning, or more specifically deep reinforcement learning (DRL), methods have been proposed widely to address these issues. By incorporating deep learning into traditional RL, DRL is highly capable of solving complex, dynamic, and especially high-dimensional cyber defense problems. This article presents a survey of DRL approaches developed for cyber security. We touch on different vital aspects, including DRL-based security methods for cyber-physical systems, autonomous intrusion detection techniques, and multiagent DRL-based game theory simulations for defense strategies against cyberattacks. Extensive discussions and future research directions on DRL-based cyber security are also given. We expect that this comprehensive review provides the foundations for and facilitates future studies on exploring the potential of emerging DRL to cope with increasingly complex cyber security problems.

Automatic modulation classification (AMC), which aims to blindly identify the modulation type of an incoming signal at the receiver in wireless communication systems, is a fundamental signal processing technique in the physical layer to improve the spectrum utilization efficiency. Motivated by deep learning (DL) high-impact success in many informatics domains, including radio signal processing for communications, numerous recent AMC methods exploiting deep networks have been proposed to overcome the existing drawbacks of traditional approaches. DL is capable of learning the underlying characteristics of radio signals effectively for modulation pattern recognition, which in turn improves the modulation classification performance under the presence of channel impairments. In this work, we first provide the fundamental concepts of various architectures, such as neural networks, recurrent neural networks, long short-term memory, and convolutional neural networks as the necessary background. We then convey a comprehensive study of DL for AMC in wireless communications, where technical analysis is deliberated in the perspective of state-of-the-art deep architectures. Remarkably, several sophisticated structures and advanced designs of convolutional neural networks are investigated for different data types of sequential radio signals, spectrum images, and constellation images to deal with various channel impairments. Finally, we discuss some primary research challenges and potential future directions in the area of DL for modulation classification.

Situation awareness (SA) is an important constituent in human information processing and essential in pilots’ decision making processes. Acquiring and maintaining appropriate levels of SA is critical in aviation environments as it affects all decisions and actions taking place in flights and air traffic control. This paper provides an overview of recent measurement models and approaches to establishing and enhancing SA in aviation environments. Many aspects of SA are examined including the classification of SA techniques into six categories, and different theoretical SA models from individual, to shared or team, and to distributed or system levels. Quantitative and qualitative perspectives pertaining to SA methods and issues of SA for unmanned vehicles are also addressed. Furthermore, future research directions regarding SA assessment approaches are raised to deal with shortcomings of the existing state-of-the-art methods in the literature.