Path planning is one of the key technologies for autonomous flight of Unmanned Aerial Vehicle. Traditional path planning algorithms have some limitations and deficiencies in the complex and dynamic environment. In this article, we propose a deep reinforcement learning approach for three-dimensional path planning by utilizing the local information and relative distance without global information. UAV can obtain the limited environmental information nearby in the actual scenario with limited sensor capabilities. Therefore, path planning can be formulated as a Partially Observable Markov Decision Process. The recurrent neural network with temporal memory is constructed to address the partial observability problem by extracting crucial information from historical state-action sequences. We develop an action selection strategy that combines the current reward value and the state-action value to reduce the meaningless exploration. In addition, we construct two sample memory pools and propose an adaptive experience replay mechanism based on the frequency of failure. The simulation experiment results show that our method has significant improvements over Deep Q-Network and Deep Recurrent Q-Network in terms of stability and learning efficiency. Our approach successfully plans a reasonable three-dimensional path in the large-scale and complex environment, and has the perfect ability to avoid obstacles.in the unknown environment.

As a sampling-based pathfinding algorithm, Rapidly Exploring Random Trees (RRT) has been widely used in motion planning problems due to the ability to find a feasible path quickly. However, the RRT algorithm still has several shortcomings, such as the large variance in the search time, poor performance in narrow channel scenarios, and being far from the optimal path. In this paper, we propose a new RRT-based path find algorithm, Fast-RRT, to find a near-optimal path quickly. The Fast-RRT algorithm consists of two modules, including Improved RRT and Fast-Optimal. The former is aims to quickly and stably find an initial path, and the latter is to merge multiple initial paths to obtain a near-optimal path. Compared with the RRT algorithm, Fast-RRT shows the following improvements: (1) A Fast-Sampling strategy that only samples in the unreached space of the random tree was introduced to improve the search speed and algorithm stability; (2) A Random Steering strategy expansion strategy was proposed to solve the problem of poor performance in narrow channel scenarios; (3) By fusion and adjustment of paths, a near-optimal path can be faster found by Fast-RRT, 20 times faster than the RRT* algorithm. Owing to these merits, our proposed Fast-RRT outperforms RRT and RRT* in both speed and stability during experiments.

Exploring the widespread applications of unmanned aerial vehicles (UAVs) in Internet of Things has become a current research hotspot. In some tasks related to UAV-based environmental monitoring and transportation, the simultaneous consideration of UAV task allocation and path planning constitutes a category of joint optimization problems. This paper focuses on a warehouse cargo inspection scenario with multiple heterogeneous UAVs. In such scenarios, existing heuristic path finding algorithms that consider task allocation cannot make a good balance between solution time and solution quality. Therefore, in this paper, we propose a reinforcement learning assisted task allocation and conflict-free path framework to achieve better task allocation and path finding results. The framework uses a multiple traveling salesman transformation algorithm for task allocation and a multi-agent reinforcement learning (MARL) algorithm for conflict-free path finding. The path finding policy can be extended to larger-scale environments with more UAVs. We conduct the training of the path planning module and the verification of the overall framework in random environments. Simulation results show that our reinforcement learning assisted framework has a significant advantage over the existing algorithms in terms of solution time, solution quality and scalability.

In this letter, a planar loop-monopole reconfigurable antenna is proposed for mobile handsets with a relative compact dimension of 60100.8 mm3. A novel coverage strategy is proposed to utilize the loop mode for the low and the high bands (LTE700, GSM850, GSM900; LTE2300, WLAN, LTE2500), and the strip monopole mode for the medium band (DCS, PCS, UMTS). As a result, a nona-band coverage for 2G/3G/4G bands is achieved within limited space. Only one p-i-n diode is used with simple bias circuit and to avoid high insertion loss. With the proposed compact structure, the antenna is suitable for wideband smartphone applications. A prototype of the proposed antenna is fabricated and measured to verify the design strategy. The measured results match well with simulation with acceptable radiation efficiency in the covered bands. Especially, the efficiency in the low band is higher than 70%.

In this communication, a compact sabre-like antenna with low wind drag is proposed for onboard communication application, under the requirements of polarization diversity and omnidirectional coverage. The proposed antenna consists of a thin resonant cavity for omnidirectional horizontal polarization, and a folded slot is co-located with the cavity to provide omnidirectional vertical polarization. Isolation lower than -20 dB between two feeding ports is achieved in a quite compact dimension of 30 x 29 x 9 mm3 (0.24λ0 x 0.24λ0 x 0.07λ0, λ0 is the free-space wavelength at 2.44 GHz), also with the omnidirectional patterns for both polarizations. A prototype of the proposed antenna is fabricated and tested to verify the design strategy. Compared with previous designs, the cross section area for wind drag is reduced for more than 28.5%, exhibiting potential usage for high-speed moving antenna carriers.

In order to solve the problem that the existing reinforcement learning algorithm is difficult to converge due to the excessive state space of the three-dimensional path planning of the unmanned aerial vehicle, this article proposes a reinforcement learning algorithm based on the heuristic function and the maximum average reward value of the experience replay mechanism. The knowledge of track performance is introduced to construct heuristic function to guide the unmanned aerial vehicles’ action selection and reduce the useless exploration. Experience replay mechanism based on maximum average reward increases the utilization rate of excellent samples and the convergence speed of the algorithm. The simulation results show that the proposed three-dimensional path planning algorithm has good learning efficiency, and the convergence speed and training performance are significantly improved.