For metal surface defect inspection, deep-learning-based methods have largely improved the inspection accuracy. However, insufficient data and the diversity of defects usually pose challenges for these methods. To solve these problems, traditional data augmentation methods often augment data by applying image-level geometric variations, usually without introducing new features of unknown defects, which yields limited improvements in defect inspection. Given such circumstances, in this article, a new data augmentation algorithm named Mask2Defect is proposed. Via prior knowledge-based data infusing, this method is able to generate defects with varied features. A large volume of defects with different shapes, severities, scales, rotation angles, spatial locations, and part numbers can be generated in a controllable manner. These generated defects will work as teacher samples to fine-tune the inspection model and automatically adapt it to a wider range of defects. To be specific, we first encode the prior knowledge into the teacher mask via the industrial prior knowledge encoder and render the defect details according to the mask with the mask-to-defect construction network. Then, the fake-to-real domain transformation GAN is used to transform the rendered samples from the fake domain into the real defect domain. Experiments reveal that the synthesized image quality of our method outperforms the state-of-the-art generative methods, and the performance of the inspection model in defect classification and localization has also been improved by fine-tuning with the generated samples.
11 June 2020 in IEEE Transactions on Instrumentation and Measurement
Visual surface defect inspection for metal part has become a rapidly developing research field within the last decade. But due to the variances of defect shapes and scales, the inspection of tiny and irregular shape defects has posed challenges on the robustness of the inspection model. In this context, a deep-learning method based on the Deformable Convolution and Concatenate Feature Pyramid Neural Networks is proposed to improve the inspection. We design a deformable convolution layer in the neural networks as an attention mechanism to adaptively extract the features of defect shape and location, which enhances the inspection of the defects with large shape variances. We also merge the multiple hierarchical features collected from different deformable convolution layers by the concatenate feature pyramid, which improves the inspection of tiny defects. The results show that the proposed method has a better generalization ability than traditional convolution neural networks.
Occlusion-aware vehicle detection in UAV imagery is challenging due to partial visibility from varied viewpoints, dense scenes, and limited features. To address this, we introduce two contributions. First, VOD-UAV, the first UAV-based vehicle detection dataset focused on occlusion, containing 712 synthetic and 1219 real-world images, each annotated with five discrete occlusion levels. These fine-grained labels enable structured supervision and detailed analysis under varying visibility conditions. Second, DOMino-YOLO, a YOLOv11-based detection framework, enhances occlusion robustness via three components: the Deformable Convolution Enhanced Module (DCEM) for spatial alignment, the Visibility-Aware Structural Aggregation (VASA) module for multi-scale feature extraction from partially visible regions, and the Context-Suppressed Implicit Modulation Head (CSIM-Head) for reducing false activations by adaptive channel reweighting. An Occlusion-Aware Repulsion Loss (OAR-Loss) combines Repulsion Loss and Visibility-Weighted Classification Loss to suppress redundant predictions and emphasize heavily occluded objects. Extensive experiments on VOD-UAV demonstrate that DOMino-YOLO significantly improves detection accuracy and robustness under occlusion. The dataset and code will publicly available to support future research.
3 September 2024 in IEEE Transactions on Instrumentation and Measurement
For infrared target segmentation, Convolutional Neural Networks based (CNN-based) methods have largely improved the segmentation accuracy. Previous CNN-based methods typically improved the segmentation accuracy by enhancing and aggregating multi-level features. But the existence of semantic gap between features from different semantic levels limits the improvement of feature aggregation. To address this problem and improve the segmentation accuracy of infrared targets with tiny sizes and irregular shapes, the Multi-Attention Shape Adaptive Network (MASANet) is proposed in this paper. Specifically, a Spatially Feature Fusion Dense-nested Unet (SFFDUnet) is designed as the feature extraction and aggregation backbone. Based on SFFDUnet, the Bi-directional Shape Adaptive Attention with Channel-wise Shape Enhance (BSAA-CSE) block is proposed as a multi-attention mechanism to reduce the semantic gap between features. Moreover, the multi-scale multi-branch deep supervision is introduced to stabilize the training process and enable model pruning. In addition, a new infrared target dataset CITY-IRTD is also constructed to evaluate the performance of our method. Experiments demonstrate the effectiveness of our method in reducing the semantic gap between features to be fused. As a result, our MASANet has achieved better performance on both public and our self-developed datasets, compared to other state-of-the-art infrared target segmentation methods. Codes and the dataset will be released at https://github.com/twentyfiveYang/MASANet_CITY-IRTD.