Unmanned aerial vehicles (UAVs) have become important tools for power transmission line inspection. Cameras installed on the platforms can efficiently obtain aerial images containing information about power equipment. However, most of the existing inspection systems cannot perform automatic real-time detection of transmission line components. In this paper, an automatic transmission line inspection system incorporating UAV remote sensing with binocular visual perception technology is developed to accurately detect and locate power equipment in real time. The system consists of a UAV module, embedded industrial computer, binocular visual perception module, and control and observation module. Insulators, which are key components in power transmission lines as well as fault-prone components, are selected as the detection targets. Insulator detection and spatial localization in aerial images with cluttered backgrounds are interesting but challenging tasks for an automatic transmission line inspection system. A two-stage strategy is proposed to achieve precise identification of insulators. First, candidate insulator regions are obtained based on RGB-D saliency detection. Then, the skeleton structure of candidate insulator regions is extracted. We implement a structure search to realize the final accurate detection of insulators. On the basis of insulator detection results, we further propose a real-time object spatial localization method that combines binocular stereo vision and a global positioning system (GPS). The longitude, latitude, and height of insulators are obtained through coordinate conversion based on the UAV’s real-time flight data and equipment parameters. Experiment results in the actual inspection environment (220 kV power transmission line) show that the presented system meets the requirement of robustness and accuracy of insulator detection and spatial localization in practical engineering.

The existing unmanned aerial vehicle (UAV)-based electric transmission line inspection systems generally adopt manual control and follow the predefined path, which reduces the efficiency and makes a high inspection cost. In this article, a UAV system with advanced embedded processors and binocular visual sensors is developed to generate guidance information from power lines in real-time and achieve automatic transmission line inspection. To realize 3-D autonomous perception of power lines, we first propose an end-to-end convolutional neural network (CNN) to extract complementary information from multilevel features and detect power lines with different pixel widths and orientations. Specifically, multilevel feature aggregation module fuses multilevel features within the same stage by learning the weight vector related to the content. The joint attention (JA) module is proposed to extract rich semantic information and suppress the background noises. Meanwhile, multistage detection results are fused to enhance the robustness of the proposed network. Subsequently, power lines are grouped according to the morphological characteristics of thinning detection results, and 3-D point sets of power lines are constructed based on the epipolar constraint of binocular images. Finally, the target point of current stereo images is generated by projecting 3-D power line points to the horizontal and vertical planes. The few-waypoint trajectory is generated based on continuous target points, and automatic inspection is finished with the proposed real-time motion planning strategy. Experimental results on four datasets show that the proposed power line detection method outperforms other state-of-the-art methods. The developed UAV platform and the proposed autonomous inspection strategy are evaluated in practical environments to validate the robustness and effectiveness.

In recent years, autonomous underwater vehicle (AUV) swarms are gradually becoming popular and have been widely promoted in ocean exploration or underwater tracking, etc. In this paper, we propose a multi-AUV cooperative underwater multi-target tracking algorithm especially when the real underwater factors are taken into account. We first give normally modelling approach for the underwater sonar-based detection and the ocean current interference on the target tracking process. Then, based on software-defined networking (SDN), we regard the AUV swarm as a underwater ad-hoc network and propose a hierarchical software-defined multi-AUV reinforcement learning (HSARL) architecture. Based on the proposed HSARL architecture, we propose the ”Dynamic-Switching” mechanism, it includes ”Dynamic-Switching Attention” and ”Dynamic-Switching Resampling” mechanisms which accelerate the HSARL algorithm's convergence speed and effectively prevents it from getting stuck in a local optimum state. Additionally, we introduce the reward reshaping mechanism for further accelerating the convergence speed of the proposed HSARL algorithm in early phase. Finally, based on a proposed AUV classification method, we propose a cooperative tracking algorithm called D ynamic- S witching- B ased M ARL (DSBM)-driven tracking algorithm. Evaluation results demonstrate that our proposed DSBM tracking algorithm can perform precise underwater multi-target tracking, comparing with many of recent research products in terms of various important metrics.

Existing size measurement systems cannot meet the requirements of non-contact measurement tasks, and the main challenge is how to detect various objects and improve measure accuracy. In order to solve these problems, a novel measurement system was proposed in this paper. In the system, a three-camera model with variable baselines was designed based on the trinocular vision. Three binocular vision subsystems were composed of the three cameras, which were used to obtain depth information from different shooting angles, and the baselines between cameras could be adjusted according to different objects. In measurement process, target object was detected automatically based on visual saliency features and spatial information. Finally, the size of target object was computed by cooperative analysis of the three binocular vision subsystems. The experimental results demonstrated that the proposed system is accurate and stable in various objects detection and measurement tasks.

Overhead transmission line corridor detection is important to ensure the safety of power facilities. Frequent and uncertain changes in the transmission line corridor environment requires an efficient and autonomous UAV inspection system, whereas the existing UAV-based inspection systems has some shortcomings in control model and ground clearance measurement. For one thing, the existing manual control model has the risk of striking power lines because it is difficult for manipulators to judge the distance between the UAV fuselage and power lines accurately. For another, the ground clearance methods based on UAV usually depend on LiDAR (Light Detection and Ranging) or single-view visual repeat scanning, with which it is difficult to balance efficiency and accuracy. Aiming at addressing the challenging issues above, a novel UAV inspection system is developed, which can sense 3D information of transmission line corridor by the cooperation of the dual-view stereovision module and an advanced embedded NVIDIA platform. In addition, a series of advanced algorithms are embedded in the system to realize autonomous control of UAVs and ground clearance measurement. Firstly, an edge-assisted power line detection method is proposed to locate the power line accurately. Then, 3D reconstruction of the power line is achieved based on binocular vision, and the target flight points are generated in the world coordinate system one-by-one to guide the UAVs movement along power lines autonomously. In order to correctly detect whether the ground clearances are in the range of safety, we propose an aerial image classification based on a light-weight semantic segmentation network to provide auxiliary information categories of ground objects. Then, the 3D points of ground objects are reconstructed according to the matching points set obtained by an efficient feature matching method, and concatenated with 3D points of power lines. Finally, the ground clearance can be measured and detected according to the generated 3D points of the transmission line corridor. Tests on both corresponding datasets and practical 220-kV transmission line corridors are conducted. The experimental results of different modules reveal that our proposed system can be applied in practical inspection environments and has good performance.

It is an essential task to inspect ground clearance of transmission lines in time. However, the weak texture of transmission lines and high complexity of the background make it difficult to balance efficiency and accuracy. To solve the problem, a trinocular vision and spatial prior based method is proposed, which is specifically designed for ground clearance measurement of transmission lines with unmanned aerial vehicles (UAVs). In this novel method, a perpendicular double-baseline trinocular vision module is applied to improve the accuracy of transmission line reconstruction. Then the spatial prior information of geometric models under different shooting attitudes is analyzed in detail, and it is adopted to determine the ground crossing points and compute ground clearance efficiently. Also, an interactive software is developed and tested in the simulation environment of UAV inspection. Experimental results verify the feasibility of the measurement method. Finally, we discuss in detail how to apply the method effectively in practice and give a set of recommended camera parameters.