12 April 2019 in IEEE Sensors Journal
Vision-based automatic railway fastener inspection, instead of manual operation, remains a great challenge. Even though many supervised learning-based methods have been developed, expensive training labels and imbalanced data are the main obstacles to leverage the performance of the fastener inspection task. To tackle the problems, we present a novel vision-based fastener inspection system (VFIS) which is inspired by few-shot learning. VFIS can automatically collect and annotate a large number of fastener samples by using the proposed online template matching-based classification method, and it only requires a very small number of annotated fastener templates. Moreover, we employ a similarity-based deep network to solve the problem of the imbalanced dataset. The comprehensive experiments are conducted on a large scale fastener dataset. VFIS yields competitive performance on both fastener localization and fastener classification. Specifically, an average detection rate of 99.36% is achieved for fastener localization, and an average accuracy of 92.69% is achieved for fastener classification. Moreover, for identifying defective fasteners, our proposed method achieves an average precision of 92.63% and an average recall of 92.88%.
21 December 2022 in International Journal of Machine Learning and Cybernetics
30 September 2022 in ACM Transactions on Multimedia Computing, Communications, and Applications
Detecting out-of-distribution (OOD) inputs for deep learning models is a critical task when models are deployed in real-world environments. Recently, a large number of works have been dedicated to tackling the OOD detection problem. One of the most straightforward and effective ways is OOD training, which adds heterogeneous auxiliary data in the training stage. However, the extra auxiliary data cannot be involved arbitrarily. A high-quality and powerful auxiliary dataset must contain samples that belong to OOD but are close to in-distribution (ID), which can teach the model to learn more information about OOD samples, furthermore, distinguish OOD from ID. The key issue for this problem is how to simply acquire such distinctive OOD samples. In this article, we propose an enhanced Mixup-based OOD (MixOOD) detection strategy that can be attached to any threshold-based OOD detecting method. Different from the traditional Mixup designed for ID data augmentation, our proposed MixOOD generates augmented images with deliberately modified Mixup and then uses them as auxiliary OOD data to leverage the OOD detection. We test our method with classical OOD detecting approaches like Maximum Softmax Probability, Energy Score, and Out-of-distribution detector for Neural networks. Experiments show that models with MixOOD can better distinguish in- and out-of-distribution samples than the original version of each approach.
3 May 2022 in Railway Sciences
Purpose: This research aims to improve the performance of rail fastener defect inspection method for multi railways, to effectively ensure the safety of railway operation.Design/methodology/approach: Firstly, a fastener region location method based on online learning strategy was proposed, which can locate fastener regions according to the prior knowledge of track image and template matching method. Online learning strategy is used to update the template library dynamically, so that the method not only can locate fastener regions in the track images of multi railways, but also can automatically collect and annotate fastener samples. Secondly, a fastener defect recognition method based on deep convolutional neural network was proposed. The structure of recognition network was designed according to the smaller size and the relatively single content of the fastener region. The data augmentation method based on the sample random sorting strategy is adopted to reduce the impact of the imbalance of sample size on recognition performance.Findings: Test verification of the proposed method is conducted based on the rail fastener datasets of multi railways. Specifically, fastener location module has achieved an average detection rate of 99.36%, and fastener defect recognition module has achieved an average precision of 96.82%.Originality/value: The proposed method can accurately locate fastener regions and identify fastener defect in the track images of different railways, which has high reliability and strong adaptability to multi railways.
1 March 2019 in Applied Intelligence