Recently, unmanned aerial vehicles (UAVs) communications gained significant concentration as a talented technology for future wireless communications using its remarkable advantages and broad applicability. Furthermore, UAV networks’ high complex configurations and designs encourage researchers to leverage relevant artificial intelligence (AI) techniques for better beyond fifth-generation (B5G)/sixth-generation (6G) services. This article summarizes AI-aided UAV solutions designated for forthcoming wireless networks. Besides, we deliver a comprehensive summary of machine learning (ML) approaches, including their applications and valuable contributions towards effective UAV network implementations, particularly advanced ML ones like bandits, federated learning (FL), meta-learning, etc. Finally, detailed UAV communication-related future research scopes and challenges is highlighted.

Reconfigurable intelligent surface (RIS) and unmanned aerial vehicle (UAV) are anticipated as talented technologies to extend the range of millimeter wave (mmWave) communications. In this letter, a UAV equipped with RIS (UAV-RIS) is used to assist mmWave base station (BS) in covering users in hotspot areas. In this context, UAV should cover several high-capacity hotspots while minimizing its flying/hovering energy consumptions. Energy-aware multi-armed bandit (EA-MAB) algorithm is proposed as an effective online learning tool to handle this problem efficiently. By which, the UAV acts as the player trying to maximize its achievable rate, i.e., the reward, over selecting different hotspots in its trajectory, i.e., the arms of the bandit game. This is done while minimizing the energy/cost of the UAV flight from one hotspot to another over the time span of its battery life. Numerical analysis confirms the superior performance of the proposed EA-MAB algorithm over benchmarks.

Millimeter wave (mmWave) communication has attracted considerable attention as a key technology for the next-generation wireless communications thanks to its exceptional advantages. MmWave leads the way to achieve a high transmission quality with directed narrow beams from source to multiple destinations by adopting different antenna beamforming (BF) techniques, which have a pivotal role in establishing and maintaining robust links. However, realizing such BF gains in practice requires overcoming several challenges, such as severe signal deterioration, hardware constraints, and design complexity. The elevated complexity of configuring mmWave BF vectors encourages researchers to leverage relevant machine learning (ML) techniques for better BF configurations deployment in 5G and beyond. In this article, we summarize mmWave BF strategies employed for future wireless networks. Then, we provide a comprehensive overview of ML techniques plus its applications and promising contributions toward efficient mmWave BF deployment. Furthermore, we discuss mmWave BF’s future research directions and challenges. Finally, we discuss a single and concurrent mmWave BF case study by applying multiarmed bandit to confirm the superiority of ML-based methods over conventional ones.

The convergence of sixth-generation (6G) networks with unmanned aerial vehicles (UAVs) and satellites is poised to introduce substantial improvements to the landscape of wireless communication, paving the way for a unified and uninterrupted space-air-ground-sea network that ensures comprehensive global connectivity. At the heart of this transformative paradigm lies artificial intelligence (AI), which drives innovation across diverse sectors by enhancing decision-making autonomy, enabling real-time data processing, and optimizing network performance and coverage. This survey paper explores AI-enabled UAV-satellite communications for 6G applications, focusing on its challenges, potential, and future. This new system combines the strengths of 6G networks, UAVs (advanced drones), and satellites. It opens up new possibilities in precision agriculture, disaster management, enhanced telecommunication services, and remote sensing. Despite its promise, this field faces complex challenges. These include spectrum management, security risks, regulatory barriers, and integrating AI operations seamlessly. This paper comprehensively analyzes these challenges, offering innovative solutions and outlining future research directions to unlock the complete capabilities of 6G-enabled UAV-satellite communications. Furthermore, it includes a case study demonstrating the effectiveness of multi-armed bandit (MAB) algorithms in optimizing resource allocation and decision-making processes for UAV-low Earth orbit (LEO) satellite communication scenarios, showcasing significant improvements in network performance. This work lays the foundation for a new generation of ultra-connected, data-driven applications that will redefine global connectivity and technological advancement by addressing these critical aspects.

Device-to-device (D2D) communication is a promising paradigm for the fifth generation (5G) and beyond 5G (B5G) networks. Although D2D communication provides several benefits, including limited interference, energy efficiency, reduced delay, and network overhead, it faces a lot of technical challenges such as network architecture, and neighbor discovery, etc. The complexity of configuring D2D links and managing their interference, especially when using millimeter-wave (mmWave), inspire researchers to leverage different machine-learning (ML) techniques to address these problems towards boosting the performance of D2D networks. In this paper, a comprehensive survey about recent research activities on D2D networks will be explored with putting more emphasis on utilizing mmWave and ML methods. After exploring existing D2D research directions accompanied with their existing conventional solutions, we will show how different ML techniques can be applied to enhance the D2D networks performance over using conventional ways. Then, still open research directions in ML applications on D2D networks will be investigated including their essential needs. A case study of applying multi-armed bandit (MAB) as an efficient online ML tool to enhance the performance of neighbor discovery and selection (NDS) in mmWave D2D networks will be presented. This case study will put emphasis on the high potency of using ML solutions over using the conventional non-ML based methods for highly improving the average throughput performance of mmWave NDS.

Millimeter wave (mmWave) band, i.e., 30 to 300 GHz, is characterized by short range transmissions and vulnerability to path blockage necessitating the use of relaying. Probing more relays finds out the best relay having the highest spectral efficiency but at the expense of increasing the probing overhead due to excessive beamforming training (BT) causing a decrease in the overall throughput. In this paper, mmWave two-hop relaying will be formulated as a single player multi-armed bandit (MAB) problem enabling one relay probing while maximizing the achievable spectral efficiency. Moreover, the relays could not establish the mmWave link due to blockage for instance will be identified as sleeping relays and eliminated from the rest of the MAB game. Thus, sleeping non contextual MAB (S-MAB) algorithm, namely sleeping Thompson sampling (S-TS) will be proposed to handle the problem. Furthermore, by utilizing the multiband capability of standardized WiGig devices containing both 2.4/5 GHz WiFi and 60 GHz mmWave bands, WiFi information will be used as contexts of the MAB game. Therefore, sleeping contextual MAB (S-CMAB) algorithm, namely S-CTS, will be proposed as well. Numerical and regret analysis ensure the superior performance of the S-CMAB algorithm over the S-MAB counterpart and the existing mmWave relay probing solutions accompanied with high convergence rates.