Unmanned aerial vehicles (UAVs) have become important tools for power transmission line inspection. Cameras installed on the platforms can efficiently obtain aerial images containing information about power equipment. However, most of the existing inspection systems cannot perform automatic real-time detection of transmission line components. In this paper, an automatic transmission line inspection system incorporating UAV remote sensing with binocular visual perception technology is developed to accurately detect and locate power equipment in real time. The system consists of a UAV module, embedded industrial computer, binocular visual perception module, and control and observation module. Insulators, which are key components in power transmission lines as well as fault-prone components, are selected as the detection targets. Insulator detection and spatial localization in aerial images with cluttered backgrounds are interesting but challenging tasks for an automatic transmission line inspection system. A two-stage strategy is proposed to achieve precise identification of insulators. First, candidate insulator regions are obtained based on RGB-D saliency detection. Then, the skeleton structure of candidate insulator regions is extracted. We implement a structure search to realize the final accurate detection of insulators. On the basis of insulator detection results, we further propose a real-time object spatial localization method that combines binocular stereo vision and a global positioning system (GPS). The longitude, latitude, and height of insulators are obtained through coordinate conversion based on the UAV’s real-time flight data and equipment parameters. Experiment results in the actual inspection environment (220 kV power transmission line) show that the presented system meets the requirement of robustness and accuracy of insulator detection and spatial localization in practical engineering.

The existing unmanned aerial vehicle (UAV)-based electric transmission line inspection systems generally adopt manual control and follow the predefined path, which reduces the efficiency and makes a high inspection cost. In this article, a UAV system with advanced embedded processors and binocular visual sensors is developed to generate guidance information from power lines in real-time and achieve automatic transmission line inspection. To realize 3-D autonomous perception of power lines, we first propose an end-to-end convolutional neural network (CNN) to extract complementary information from multilevel features and detect power lines with different pixel widths and orientations. Specifically, multilevel feature aggregation module fuses multilevel features within the same stage by learning the weight vector related to the content. The joint attention (JA) module is proposed to extract rich semantic information and suppress the background noises. Meanwhile, multistage detection results are fused to enhance the robustness of the proposed network. Subsequently, power lines are grouped according to the morphological characteristics of thinning detection results, and 3-D point sets of power lines are constructed based on the epipolar constraint of binocular images. Finally, the target point of current stereo images is generated by projecting 3-D power line points to the horizontal and vertical planes. The few-waypoint trajectory is generated based on continuous target points, and automatic inspection is finished with the proposed real-time motion planning strategy. Experimental results on four datasets show that the proposed power line detection method outperforms other state-of-the-art methods. The developed UAV platform and the proposed autonomous inspection strategy are evaluated in practical environments to validate the robustness and effectiveness.

Real-time scene comprehension is the basis for automatic electric power inspection. However, existing RGB-based scene comprehension methods may achieve unsatisfied performance when dealing with complex scenarios, insufficient illumination or occluded appearances. To solve this problem, by cooperating visual and thermal images, the Dual-Space Graph-based Interaction Network (DSGBINet) is proposed to achieve all-day time semantic segmentation of power equipment in high-voltage power transmission line and electric transformer substation scenes. Specifically, modality-specific features are first extracted via two separate backbone networks with the same architecture. Multi-modality high-level features are first fused via long-range relationship in coordinate space. Then, multi-modality features from regular grids are further clustered and assigned to vertices in feature space. Cross-graph and inner-graph regional relations are utilized for reasoning and enhancement, which could exploit the mutual benefits and extract rich contextual information in a semantic view. Furthermore, to overcome the huge scale difference and the inherent characteristic of thermal images, the idea of multi-task learning is integrated into the decoding process. The edge detection and semantic segmentation are achieved collaboratively, which could segment the different power equipment more accurately and completely. The comparative and ablation experiments on the proposed two RGB-T semantic segmentation datasets evaluate the effectiveness and robustness of the proposed network compared with existing state-of-the-art methods. The extended experiments on the public datasets further demonstrate the superiority of the proposed method. Our dataset and code will be released at: https://github.com/hhujiang/DSGBINet.

As a typical deep-learning model, Convolutional Neural Networks (CNNs) can be exploited to automatically extract features from images using the hierarchical structure inspired by mammalian visual system. For image classification tasks, traditional CNN models employ the softmax function for classification. However, owing to the limited capacity of the softmax function, there are some shortcomings of traditional CNN models in image classification. To deal with this problem, a new method combining Biomimetic Pattern Recognition (BPR) with CNNs is proposed for image classification. BPR performs class recognition by a union of geometrical cover sets in a high-dimensional feature space and therefore can overcome some disadvantages of traditional pattern recognition. The proposed method is evaluated on three famous image classification benchmarks, that is, MNIST, AR, and CIFAR-10. The classification accuracies of the proposed method for the three datasets are 99.01%, 98.40%, and 87.11%, respectively, which are much higher in comparison with the other four methods in most cases.

Machine vision inspection technology provides an efficient tool for surface defects inspection. However, because of the multiformity of surface defects, the existing machine vision methods for surface defects inspection are limited by application scenarios. In order to improve the versatility of algorithms, and to process various kinds of images more accurately, we propose a new adaptive method for surface defect detection, named neighborhood gray-level difference method using the multidirectional gray-level fluctuation. This method changes thresholds and step values by extracting gray-level-fluctuating condition of images, and then it uses the neighborhood gray-level difference to segment defects from background. Experimental results demonstrate the effectiveness of the proposed method for inspecting different surface defects. Compared with other methods, the proposed method can be applied to inspect various surface defects, and it can provide more accurate defect segmentation results.