Surface defect inspection of no-service rail is important for safety of railway transportation. However, there are several challenges of irregular defect boundary, similar foreground and background for no-service rail surface defect inspection. To deal with the above challenges, depth image is used to provide complementary spatial information to RGB image. In recent years, with the development of deep learning and computer vision technology, intelligent inspection of defect has made great progress. We propose a neural network named collaborative learning attention network (CLANet) for no-service rail surface defect inspection. Our method can inspect the defect object of rail surface and segment the accurate region of that defect. The proposed method consists of three main stages: feature extraction, cross-modal information fusion, and defect location and segmentation. A multimodal attention block is proposed to highlight complex defect object with a new cross-modal fusion strategy. Furthermore, dual stream decoder enriches the representation of advanced features and avoids the dilution of information in the decoding stage. Suffering from the scarcity of defective data, an industrial RGB-D dataset NEU RSDDS-AUG is built. Finally, ablation studies verify the effectiveness of our proposed method. Compared with the existing nine state-of-the-art methods, CLANet has achieved improvements in all five parameters. Our method is also competitive on four public benchmark datasets.

Surface defect segmentation of no-service rail is important for its quality assessment. There are several challenges of uneven illumination, complex background, and difficulty of sample collection for no-service rail surface defects (NRSDs). In this paper, we propose an acquisition scheme with two lamp light and color scan line CCD to alleviate uneven illumination. Then a multiple context information segmentation network is proposed to improve NRSD segmentation. The network makes full use of context information based on dense block, pyramid pooling module and multi-information integration. Besides, the attention mechanism is applied to optimize extracted information by filtering noise. For the problem of real sample shortage, we propose to utilize artificial samples to train network. And a NRSD dataset NRSD-MN is built with artificial NRSDs and natural NRSDs. Experiment results show that our method is feasible and has a good segmentation effect on artificial and natural NRSDs.

RGB and thermal infrared (RGBT) image saliency detection is a relatively new direction in the field of computer vision. Combining the advantages of RGB images and T images can significantly improve detection performance. Currently, there are only a few methods to work on RGBT saliency detection, and the number of image samples cannot meet the training requirements for deep learning, so it remains valuable to propose an effective unsupervised method. In this paper, we present an unsupervised RGBT saliency detection method based on multi-graph fusion and learning. Firstly, RGB images and T images are adaptively fused based on boundary information to produce more accurate superpixels. Next, a multi-graph fusion model is proposed to selectively learn useful information from multi-modal images. Finally, we implement the theory of finding good neighbors in the graph affinity and propose different algorithms for two stages of saliency ranking. Experimental results on three RGBT datasets show that the proposed method is effective compared with the state-of-the-art algorithms.

Deep neural networks have greatly improved the performance of rail surface defect segmentation when the test samples have the same distribution as the training samples. However, in practical inspection scenarios, the rail surface exhibits variations in appearance due to different service times and natural conditions. Conventional deep learning models show limited generalization in scenes with distribution differences. To address this problem, we propose a novel one-shot unsupervised domain adaptation framework. Specifically, we introduce a shape-consistent style transfer module that performs pixel-level distribution alignment between the training and test images. Based on the one-shot test image, the training image is reconstructed to have the same appearance as the test image. Meanwhile, we employ a multitask learning strategy to prevent content distortion of the reconstructed images. To improve the robustness of the model to distribution differences, we design an edge-aware defect segmentation model and train the model using the reconstructed training images. The experimental results show that our method effectively improves the robustness of the model to distribution differences and achieves satisfying results in the task of rail surface defect segmentation.

Anomaly detection is a crucial means to detect unbalanced and multiclass abnormal data in industrial products. The existing anomaly detection model is usually suitable for products with a simple or regular background texture. However, there is a lack of a universal surface defect anomaly detection framework for railway products when facing the challenges: the complex background texture, huge intraclass variation, uneven distribution of interclass, and weak texture-featured defects. To fill this gap, an innovative generative adversarial network based on adaptive pyramid graph (APG) and variation residuals (APGVR-GAN) is proposed, aiming to improve the robustness of anomaly detection in railway products and other complex industrial supplies. First, the APG module is embedded in the encoder–decoder–encoder pattern, capturing the correlation description between neighbor regions, which is utilized to enhance the detection of abnormal defects with weak texture. Next, the variation residual module is employed to enhance the expression of various normal samples in the latent space and improve the identification ability for abnormal samples. Then, the dual-probability prototype loss is proposed to make different normal samples have more concentrated expression and more similar probability distribution centers in latent space. Finally, an adaptive focal-gate loss and a regularized log-likelihood loss are designed to overcome the imbalance problem in training samples with different background information. The effectiveness of the model is verified on three new railway datasets and three other industrial public datasets. The results show that in anomaly detection, APGVR-GAN on the three railway datasets, Northeastern University (NEU)-the rail surface defects dataset (RSDD)-1 dataset, NEU-RSDD-1 dataset, and RSDD area under curve (AUC) [that is the receiver operating characteristic curve (ROC)], can reach 0.89, 0.81, and 0.87, and its detection accuracy (ACC) reached 0.97, 0.96, and 0.92, which is much higher than other comparative anomaly detection algorithms. In the segmentation experiments on the RSDD dataset, compared with other known rail defect detection methods, the overall evaluation metric F-Score is 83.70 which is also better than other methods.