Dynamic spectrum access (DSA) is regarded as an effective and efficient technology to share radio spectrum among different networks. As a secondary user (SU), a DSA device will face two critical problems: avoiding causing harmful interference to primary users (PUs), and conducting effective interference coordination with other secondary users. These two problems become even more challenging for a distributed DSA network where there is no centralized controllers for SUs. In this paper, we investigate communication strategies of a distributive DSA network under the presence of spectrum sensing errors. To be specific, we apply the powerful machine learning tool, deep reinforcement learning (DRL), for SUs to learn “appropriate” spectrum access strategies in a distributed fashion assuming NO knowledge of the underlying system statistics. Furthermore, a special type of recurrent neural network (RNN), called the reservoir computing (RC), is utilized to realize DRL by taking advantage of the underlying temporal correlation of the DSA network. Using the introduced machine learning-based strategy, SUs could make spectrum access decisions distributedly relying only on their own current and past spectrum sensing outcomes. Through extensive experiments, our results suggest that the RC-based spectrum access strategy can help the SU to significantly reduce the chances of collision with PUs and other SUs. We also show that our scheme outperforms the myopic method which assumes the knowledge of system statistics, and converges faster than the Q-learning method when the number of channels is large.

Unmanned aerial vehicles (UAVs) are attracting increasing attention for applications such as video streaming, surveillance, and delivery using reliable line-of-sight (LOS) links. Nevertheless, due to the large radio-frequency (RF) transmission footprint from a UAV transmitted to ground nodes, UAV communications may significantly deteriorate the performance of cochannel ground communication links. With the lack of a dedicated spectrum, researchers need to design efficient spectrum-sharing policies for UAV communications to enhance spectral efficiency (SE) and control interference-to-ground communications. One technique for spectrum sharing is spatial spectrum sensing (SSS), which enables devices to sense spatial spectrum opportunities and reuse them aggressively and efficiently by controlling the SSS radius. The goal of this article is to introduce the fundamentals, challenges, and applications of SSS for UAV spectrum access and discuss open research problems for realizing UAV spectrum sharing, including dynamic spectrum access for swarm UAV networks, artificial intelligence (AI)-enabled UAV spectrum access, blockchain-based UAV spectrum access, multichannel access for UAVs, and the integration of UAVs into cellular networks.

Reservoir computing (RC) is a class of neuromorphic computing approaches that deals particularly well with time-series prediction tasks. It significantly reduces the training complexity of recurrent neural networks and is also suitable for hardware implementation whereby device physics are utilized in performing data processing. In this paper, the RC concept is applied to detecting a transmitted symbol in multiple-input multiple-output orthogonal frequency division multiplexing (MIMO-OFDM) systems. Due to wireless propagation, the transmitted signal may undergo severe distortion before reaching the receiver. The nonlinear distortion introduced by the power amplifier at the transmitter may further complicate this process. Therefore, an efficient symbol detection strategy becomes critical. The conventional approach for symbol detection at the receiver requires accurate channel estimation of the underlying MIMO-OFDM system. However, in this paper, we introduce a novel symbol detection scheme where the estimation of the MIMO-OFDM channel becomes unnecessary. The introduced scheme utilizes an echo state network (ESN), which is a special class of RC. The ESN acts as a black box for system modeling purposes and can predict nonlinear dynamic systems in an efficient way. Simulation results for the uncoded bit error rate of nonlinear MIMO-OFDM systems show that the introduced scheme outperforms conventional symbol detection methods.

Dynamic spectrum access (DSA) has the great potential to alleviate spectrum shortage and promote network capacity. However, two fundamental technical issues have to be addressed, namely, interference coordination between DSA users and interference suppression for primary users (PUs). These two issues are very challenging since generally there is no powerful infrastructures in DSA networks to support centralized control. As a result, DSA users have to perform spectrum management individually, including spectrum access and power allocation, without accurate channel state information and centralized control. In this article, a novel spectrum management framework is proposed, in which Q-learning, a type of reinforcement learning, is utilized to enable DSA users to carry out effective spectrum management individually and intelligently. For more efficient process, neural networks (NNs) are employed to implement Q-learning processes, so-called deep Q-network (DQN). Furthermore, we also investigate the optimal way to construct DQN considering both the performance of wireless communications and the difficulty of NN training. Finally, extensive simulation studies are conducted to demonstrate the effectiveness of the proposed spectrum management framework.

Deep reinforcement learning (DRL) has been shown to be successful in many application domains. Combining recurrent neural networks (RNNs) and DRL further enables DRL to be applicable in non-Markovian environments by capturing temporal information. However, training of both DRL and RNNs is known to be challenging requiring a large amount of training data to achieve convergence. In many targeted applications, such as those used in the fifth-generation (5G) cellular communication, the environment is highly dynamic, while the available training data is very limited. Therefore, it is extremely important to develop DRL strategies that are capable of capturing the temporal correlation of the dynamic environment requiring limited training overhead. In this article, we introduce the deep echo state Q-network (DEQN) that can adapt to the highly dynamic environment in a short period of time with limited training data. We evaluate the performance of the introduced DEQN method under the dynamic spectrum sharing (DSS) scenario, which is a promising technology in 5G and future 6G networks to increase the spectrum utilization. Compared with conventional spectrum management policy that grants a fixed spectrum band to a single system for exclusive access, DSS allows the secondary system to share the spectrum with the primary system. Our work sheds light on the application of an efficient DRL framework in highly dynamic environments with limited available training data.