With the development of the smart grid, the traditional defect detection methods in transmission lines are gradually shifted to the combination of robots or drones and deep learning technology to realize the automatic detection of defects, avoiding the risks and computational costs of manual detection. Lightweight embedded devices such as drones and robots belong to small devices with limited computational resources, while deep learning mostly relies on deep neural networks with huge computational resources. And semantic features of deep networks are richer, which are also critical for accurately classifying morphologically similar defects for detection, helping to identify differences and classify transmission line components. Therefore, we propose a method to obtain advanced semantic features even in shallow networks. Combined with transfer learning, we change the image features (e.g., position and edge connectivity) under self-supervised learning during pre-training. This allows the pre-trained model to learn potential semantic feature representations rather than relying on low-level features. The pre-trained model then directs a shallow network to extract rich semantic features for downstream tasks. In addition, we introduce a category semantic fusion module (CSFM) to enhance feature fusion by utilizing channel attention to capture global and local information lost during compression and extraction. This module helps to obtain more category semantic information. Our experiments on a self-created transmission line defect dataset show the superiority of modifying low-level image information during pre-training when adjusting the number of network layers and embedding of the CSFM. The strategy demonstrates generalization on the publicly available PASCAL VOC dataset. Finally, compared with state-of-the-art methods on the synthetic fog insulator dataset (SFID), the strategy achieves comparable performance with much smaller network depths.

Small target detection has always been a hot and difficult point in the field of target detection. The existing detection network has a good effect on conventional targets but a poor effect on small target detection. The main challenge is that small targets have few pixels and are widely distributed in the image, so it is difficult to extract effective features, especially in the deeper neural network. A novel plug-in to extract location features of the small target in the deep network was proposed. Because the deep network has a larger receptive field and richer global information, it is easier to establish global spatial context mapping. The plug-in named location feature extraction establishes the spatial context mapping in the deep network to obtain the global information of scattered small targets in the deep feature map. Additionally, the attention mechanism can be used to strengthen attention to the spatial information. The comprehensive effect of the above two can be utilized to realize location feature extraction in the deep network. In order to improve the generalization of the network, a new self-distillation algorithm was designed for pre-training that could work under self-supervision. The experiment was conducted on the public datasets (Pascal VOC and Printed Circuit Board Defect dataset) and the self-made dedicated small target detection dataset, respectively. According to the diagnosis of the false-positive error distribution, the location error was significantly reduced, which proved the effectiveness of the plug-in proposed for location feature extraction. The mAP results can prove that the detection effect of the network applying the location feature extraction strategy is much better than the original network.