As the failure of power line insulators leads to the failure of power transmission systems, an insulator inspection system based on an aerial platform is widely used. Insulator defect detection is performed against complex backgrounds in aerial images, presenting an interesting but challenging problem. Traditional methods, based on handcrafted features or shallow-learning techniques, can only localize insulators and detect faults under specific detection conditions, such as when sufficient prior knowledge is available, with low background interference, at certain object scales, or under specific illumination conditions. This paper discusses the automatic detection of insulator defects using aerial images, accurately localizing insulator defects appearing in input images captured from real inspection environments. We propose a novel deep convolutional neural network (CNN) cascading architecture for performing localization and detecting defects in insulators. The cascading network uses a CNN based on a region proposal network to transform defect inspection into a two-level object detection problem. To address the scarcity of defect images in a real inspection environment, a data augmentation method is also proposed that includes four operations: 1) affine transformation; 2) insulator segmentation and background fusion; 3) Gaussian blur; and 4) brightness transformation. Defect detection precision and recall of the proposed method are 0.91 and 0.96 using a standard insulator dataset, and insulator defects under various conditions can be successfully detected. Experimental results demonstrate that this method meets the robustness and accuracy requirements for insulator defect detection.

Automatic metallic surface defect inspection has received increased attention in relation to the quality control of industrial products. Metallic defect detection is usually performed against complex industrial scenarios, presenting an interesting but challenging problem. Traditional methods are based on image processing or shallow machine learning techniques, but these can only detect defects under specific detection conditions, such as obvious defect contours with strong contrast and low noise, at certain scales, or under specific illumination conditions. This paper discusses the automatic detection of metallic defects with a twofold procedure that accurately localizes and classifies defects appearing in input images captured from real industrial environments. A novel cascaded autoencoder (CASAE) architecture is designed for segmenting and localizing defects. The cascading network transforms the input defect image into a pixel-wise prediction mask based on semantic segmentation. The defect regions of segmented results are classified into their specific classes via a compact convolutional neural network (CNN). Metallic defects under various conditions can be successfully detected using an industrial dataset. The experimental results demonstrate that this method meets the robustness and accuracy requirements for metallic defect detection. Meanwhile, it can also be extended to other detection applications.

A vision-based target-following guider for a mobile robot is presented in this paper. It consists of three parts: the visual tracking part, the target re-detection part, and the visual servo part. In the visual tracking part, the target contour band (TCB) is proposed for irregular sampling, which can improve the performance of the correlation filter-based methods. In the target re-detection part, potential targets are searched by an off-line trained detector. Then an online training module is used to determine the real target and achieve accurate positioning. The interaction matrix of point features is used in the visual servo part for motion control to maintain the relative pose between the target and the robot. The visual tracking part and the target re-detection part are tested respectively on many videos, which are proved to work well. To show the effectiveness of our method, some state of the art methods are also test. The target-following guider is evaluated on mobile robots. In our experiments, the robot can robustly follow the human target over a long distance, which strongly proved the validity of our target-following guider.

Calibration is an important premise for vision-guided robot system, especially for precise industrial operations on curved surfaces. Most of existing methods focus on one or partial union of errors including assembly error of the robot, relative pose error between the vision unit and the robot, or installation error of end effector, which is still challenging in practice. To solve this problem, this article proposes a new calibration method based on error correction matrix by which different manifestations of each error is unified. As the nonlinear equations formed by these matrices cannot be solved analytically, a numerical optimization solution based on Lie algebra is presented. Specifically, the matrices acquisition is formulated as the problem of minimizing the sum of distance deviations between actual and ideal tool center points. This problem is then solved by differentiating these matrices with a form of left multiplicative perturbation. In this way, a high precision joint calibration with multisource errors is achieved. The proposed method is verified by simulations and experiments.

Existing monocular visual simultaneous localization and mapping (SLAM) mainly focuses on point and line features, and the constraints among line features are not fully explored. In this paper, a multi-feature monocular SLAM with ORB points, lines, and junctions of coplanar lines is proposed for indoor environments. To create 3D junctions of coplanar lines, an adaptive coordinate confidence is designed to describe the coordinate stability of 2D junctions on the image plane. Based on the matched 2D junctions between the current keyframe and other two associated keyframes, a multi-view coplanarity verification is performed for 3D junction creation. Moreover, reprojection verification is conducted to update the observation of 3D junction, which is used to dynamically update the confidence of coplanarity for confidence-based local bundle adjustment. The camera pose is optimized through multiple constraints of ORB points, lines, and junctions of the coplanar lines. As a result, the performance of monocular SLAM is improved. The experiment verification shows the effectiveness of the proposed method.

Self-supervised monocular depth estimation has gained popularity due to its convenience of training network without dense ground truth depth annotation. Specifically, the multi-frame monocular depth estimation achieves promising results in virtue of temporal information. However, existing multi-frame solutions ignore the different impacts of pixels of input frame on depth estimation and the geometric information is still insufficiently explored. In this paper, a self-supervised monocular depth estimation framework with geometric prior and pixel-level sensitivity is proposed. Geometric constraint is involved through a geometric pose estimator with prior depth predictor and optical flow predictor. Further, an alternative learning strategy is designed to improve the learning of prior depth predictor by decoupling it with the ego-motion from the geometric pose estimator. On this basis, prior feature consistency regularization is introduced into the depth encoder. By taking the dense prior cost volume based on optical flow map and ego-motion as the supervising signal for feature consistency learning, the cost volume is obtained with more reasonable feature matching. To deal with the pixel-level difference of sensitivity in input frame, a sensitivity-adaptive depth decoder is built by flexibly adding a shorter path from cost volume to the final depth prediction. In this way, the back propagation of gradient to cost volume is adaptively adjusted, and an accurate depth map is decoded. The effectiveness of the proposed method is verified on public datasets.