Metal surface defect segmentation can play an important role in dealing with the issue of quality control during the production and manufacturing stages. There are still two major challenges in industrial applications. One is the case that the number of metal surface defect samples is severely insufficient, and the other is that the most existing algorithms can only be used for specific surface defects and it is difficult to generalize to other metal surfaces. In this work, a theory of few-shot metal generic surface defect segmentation is introduced to solve these challenges. Simultaneously, the Triplet-Graph Reasoning Network (TGRNet) and a novel dataset Surface Defects- 4 ⁱ are proposed to achieve this theory. In our TGRNet, the surface defect triplet (including triplet encoder and trip loss) is proposed and is used to segment background and defect area, respectively. Through triplet, the few-shot metal surface defect segmentation problem is transformed into few-shot semantic segmentation problem of defect area and background area. For few-shot semantic segmentation, we propose a method of multi-graph reasoning to explore the similarity relationship between different images. And to improve segmentation performance in the industrial scene, an adaptive auxiliary prediction module is proposed. For Surface Defects- 4 ⁱ , it includes multiple categories of metal surface defect images to verify the generalization performance of our TGRNet and adds the nonmetal categories (leather and tile) as extensions. Through extensive comparative experiments and ablation experiments, it is proved that our architecture can achieve state-of-the-art results.

Deep neural networks have greatly improved the performance of rail surface defect segmentation when the test samples have the same distribution as the training samples. However, in practical inspection scenarios, the rail surface exhibits variations in appearance due to different service times and natural conditions. Conventional deep learning models show limited generalization in scenes with distribution differences. To address this problem, we propose a novel one-shot unsupervised domain adaptation framework. Specifically, we introduce a shape-consistent style transfer module that performs pixel-level distribution alignment between the training and test images. Based on the one-shot test image, the training image is reconstructed to have the same appearance as the test image. Meanwhile, we employ a multitask learning strategy to prevent content distortion of the reconstructed images. To improve the robustness of the model to distribution differences, we design an edge-aware defect segmentation model and train the model using the reconstructed training images. The experimental results show that our method effectively improves the robustness of the model to distribution differences and achieves satisfying results in the task of rail surface defect segmentation.

Vision-based surface defect detection (SDD) for no-service aero-engine blades provides a fast and effective way to monitor product quality. Most existing detection algorithms for aero-engine blades are 1) based on CNN, including artificially designed non-maximum suppression (NMS) operations, and 2) focus on improving the detection accuracy rather than improving the inference speed and even ignoring the latter. To solve the above problems, we introduce a novel object detection paradigm, DEtection TRansformer (DETR), to design a novel network (SDD-DETR) with high accuracy for the SDD of aero-engine blades. To our knowledge, the paper is the first to introduce the DETR detector to SDD of aero-engine blades. While providing high accuracy, the inference speed of DETR remained slow due to self-attention operation and feed-forward network (FFN). Therefore, two lightweight modules have been designed for SDD of aero-engine blades: a progressive feature input multi-scale deformable attention module (PFI-MSDA) and a lightweight FFN (LW-FFN). PFI-MSDA hierarchically reduces the number of tokens input to the self-attention module, thereby reducing the time complexity of the self-attention layer. LW-FFN shrinks the complexity of multilayer perceptron. In addition, no parameter sharing of the detection head is utilized to compensate for the accuracy drop caused by the lightweight. Experiments verify that our method has the same AP and F1-score as DINO (a DETR-based detector), but our approach is lighter. Compared with DINO, the FLOPs are reduced by 113.4 ${G}$ , the inference speed is increased by 42.4%, and the runtime memory usage is reduced by 5.9 ${G}$ , which allows our method to be trained on low-end GPUs with more batch size, further improving the training efficiency. The code is available at https://github.com/VDT-2048/SDD-DETR. Note to Practitioners —The motivation for this paper is to design a high-precision and high-inference speed visual detection method for the SDD of aero-engine blades. Most high-precision vision methods are based on the transformer framework. However, its high complexity and poor compatibility in deployment environments lead to slower detection speeds. Although the application object in this paper is aero-engine blades, it is also applicable in other fields of industry, such as rail detection, plate and strip steel detection, etc. However, the method proposed cannot be supported by deployment environments such as RKNN because of the deformable attention operator, so it takes a certain amount of time to be deployed and put into practical use. Currently, the frameworks of visual and language large models are based on transformers, consistent with the framework of our method, which makes extending our approach to large visual and multi-modal models more accessible.

Anomaly detection is a crucial means to detect unbalanced and multiclass abnormal data in industrial products. The existing anomaly detection model is usually suitable for products with a simple or regular background texture. However, there is a lack of a universal surface defect anomaly detection framework for railway products when facing the challenges: the complex background texture, huge intraclass variation, uneven distribution of interclass, and weak texture-featured defects. To fill this gap, an innovative generative adversarial network based on adaptive pyramid graph (APG) and variation residuals (APGVR-GAN) is proposed, aiming to improve the robustness of anomaly detection in railway products and other complex industrial supplies. First, the APG module is embedded in the encoder–decoder–encoder pattern, capturing the correlation description between neighbor regions, which is utilized to enhance the detection of abnormal defects with weak texture. Next, the variation residual module is employed to enhance the expression of various normal samples in the latent space and improve the identification ability for abnormal samples. Then, the dual-probability prototype loss is proposed to make different normal samples have more concentrated expression and more similar probability distribution centers in latent space. Finally, an adaptive focal-gate loss and a regularized log-likelihood loss are designed to overcome the imbalance problem in training samples with different background information. The effectiveness of the model is verified on three new railway datasets and three other industrial public datasets. The results show that in anomaly detection, APGVR-GAN on the three railway datasets, Northeastern University (NEU)-the rail surface defects dataset (RSDD)-1 dataset, NEU-RSDD-1 dataset, and RSDD area under curve (AUC) [that is the receiver operating characteristic curve (ROC)], can reach 0.89, 0.81, and 0.87, and its detection accuracy (ACC) reached 0.97, 0.96, and 0.92, which is much higher than other comparative anomaly detection algorithms. In the segmentation experiments on the RSDD dataset, compared with other known rail defect detection methods, the overall evaluation metric F-Score is 83.70 which is also better than other methods.

Pavement distress detection is of significance for road maintenance and traffic safety. Manual pavement distress detection suffers from high workloads, inefficiency, low accuracy, and high cost. To replace the manual operations in the pre-filling detection with the aim to improve efficiency and reduce cost, this paper proposes a three-stage automatic inspection and evaluation system for pavement distress based on improved deep convolutional neural networks (CNNs). First, the system integrates multi-level context information from the CNN classification model to construct discriminative super-features to determine whether there is distress in the pavement image and the type of the distress, so as to achieve rapid detection of pavement distress. Then, the pavement images with distress are fed into the CNN segmentation model to highlight the distress region with pixel-wise. In the segmentation model, a novel pyramid feature extraction module and a novel guidance attention mechanism are introduced. Finally, we evaluate the degree of pavement damage according to the segmentation results of the CNN segmentation model. In the experiments, we compare our classification model and segmentation model with other state-of-the-art methods on two pavement distress datasets, and the results demonstrate that the proposed models achieve out-performance on different evaluation metrics.