With the wide application of unmanned aerial vehicles (UAVs), performing search and rescue missions autonomously in unknown environment has become an increasingly concerning issue. In this paper, we propose an adaptive conversion speed Q-Learning algorithm (ACSQL). Performing UAV missions autonomously is divided into two stages: rescue mission search stage and optimal path search stage. In the first stage, a UAV can find task points as soon as possible, and the efficiency of exploration is increased by adaptively adjusting the speed of the UAV. In the second stage, to get a secure and short path, we propose a subdomain search algorithm. Based on the above two stages, we improve state space and action space in reinforcement learning, and design a composite reward function, finally obtain the path of UAV to perform multiple search and rescue missions through this algorithm. In order to solve the problems of slow training convergence and high uncertainty, we initialize the Q-table by combining detection information of UAV sensors in first stage. Simulation results show that ACSQL algorithm can realize autonomous navigation and path planning of UAV in an unknown environment. Compared with traditional action space, the learning process of UAV converges faster and more stable, and it can converge in about 30 episodes. Compared with DDPG algorithm and IDWA algorithm in different scenarios, ACSQL algorithm has the shortest path length. Finally, ACSQL algorithm is verified by UAV simulator Airsim.

Due to the outstanding merits such as mobility, high maneuverability, and flexibility, Unmanned Aerial Vehicles (UAVs) are viable mobile power transmitters that can be rapidly deployed in geographically constrained regions. They are good candidates for supplying power to energy-limited Sensor Nodes (SNs) with Wireless Power Transfer (WPT) technology. In this paper, we investigate a UAV-enabled WPT system that transmits power to a set of SNs at unknown positions. A key challenge is how to efficiently gather the locations of SNs and design a power transfer scheme. We formulate a multi-objective optimization problem to jointly optimize these objectives: maximization of UAV's search efficiency, maximization of total harvested energy, minimization of UAV's flight energy consumption and maximization of UAV's energy utilization efficiency. To tackle these issues, we present a two-stage strategy that includes a UAV Motion Control (UMC) algorithm for obtaining the coordinates of SNs and a Dynamic Genetic Clustering (DGC) algorithm for power transfer via grouping SNs into clusters. First, the UMC algorithm enables the UAV to autonomously control its own motion and conduct target search missions. The objective is to make the energy-restricted UAV find as many SNs as feasible without any apriori knowledge of their information. Second, the DGC algorithm is used to optimize the energy consumption of the UAV by combining a genetic clustering algorithm with a dynamic clustering strategy to maximize the amount of energy harvested by SNs and the energy utilization efficiency of the UAV. Finally, experimental results show that the proposed algorithms outperform their counterparts.

As a key service of the Internet of Vehicles (IoV), vehicle advertising can capture the interest of potential customers and generate revenue for nearby merchants. However, due to the dynamics of vehicles, IoV suffers from unstable communication links between vehicles, which reduces the delivery ratio of vehicle advertising. To address this challenge and progress beyond the state-of-the-art work, this letter adopts Digital Twin (DT) to construct a social relationship model of vehicles and proposes a seed determination scheme for advertising dissemination. On this basis, a Deep Q-network (DQN)-based vehicle advertising dissemination scheme is proposed to determine the updating frequency of the seeds and further improve dissemination efficiency.

Satellite communication often faces challenges like path loss and high latency when supplementing terrestrial networks. We propose high-altitude platforms (HAPs) as relay nodes constructing a satellite-HAP-terrestrial integrated network (SHTIN) architecture to mitigate these issues. This architecture enhances link stability and provides low-latency computation offloading. We design a prediction-based task planning (TP) algorithm that enables efficient offload node selection by predicting resource demands, preventing overload or resource waste of agents. Guided by the TP algorithm, we employ a multi-agent proximal policy optimization (MAPPO) algorithm to optimize offloading policies, reducing energy consumption while maximizing resource utilization within tolerable task delays. Simulations confirm that our method outperforms existing approaches in resource efficiency and energy consumption, achieving superior overall performance.

The flocking movement is a fundamental and crucial operation in multi-UAVs systems, encompassing navigation and obstacle avoidance. However, traditional flocking algorithms typically rely on rigid rules and exhibit limited adaptability to diverse environments. Reinforcement learning (RL) effectively addresses this issue as a flexible and model-free framework. In this paper, RL techniques are utilized to achieve navigation and obstacle avoidance for a swarm of UAVs. To enhance training efficiency, we propose an improved algorithm called heuristic guides TD3 (HGTD3) by integrating heuristic guides with the Twin Delayed Deep Deterministic policy gradient (TD3), aiming to address the protracted learning periods commonly observed in traditional reinforcement learning methods. Considering the network-constrained environment, we propose NIFA: Network Interference Flocking Algorithm, a UAV flocking algorithm designed based on Sparrow Search Algorithm. NIFA can guide the losing UAV to follow the swarm and at the same time maintain the overall navigation and avoidance efficiency. Finally, we demonstrate the scalability and adaptability of HGTD3-NIFA in a simulation experiment in terms of multi-UAVs flocking and navigation.