Rail surface defects seriously affect the safety of railway systems. At present, human inspection and rail vehicle inspection are the main approaches for the detection of rail surface defects. However, there are many shortcomings to these approaches, such as low efficiency, high cost, and so on. This paper presents a novel visual inspection approach based on unmanned aerial vehicle (UAV) images, and focuses on two key issues of UAV-based rail images: image enhancement and defects segmentation. With regards to the first aspect, a novel image enhancement algorithm named Local Weber-like Contrast (LWLC) is proposed to enhance rail images. The rail surface defects and backgrounds can be highlighted and homogenized under various sunlight intensity by LWLC, due to its illuminance independent, local nonlinear and other advantages. With regards to the second, a new threshold segmentation method named gray stretch maximum entropy (GSME) is presented in this paper. The proposed GSME method emphasizes gray stretch and de-noising on UAV-based rail images, and selects an optimal segmentation threshold for defects detection. Two visual comparison experiments were carried out to demonstrate the efficiency of the proposed methods. Finally, a quantitative comparison experiment shows the LWLC-GSME model achieves a recall of 93.75% for T-I defects and of 94.26% for T-II defects. Therefore, LWLC for image enhancement, in conjunction with GSME for defects segmentation, is efficient and feasible for the detection of rail surface defects based on UAV Images.
25 July 2023 in IEEE Transactions on Intelligent Transportation Systems
Noise barriers play a critical role in reducing noise and preventing foreign object from invading railway. Noise barrier structural defects such as rusted column, deteriorated mortar layer and other damages make its structure unstable, thereby threatening seriously railway operation safety. Unfortunately, existing noise barrier inspection methods still rely heavily on manual inspection, which are low-efficiency, subjective and difficult to detect the external structure of noise barriers. To solve these problems, this study proposes an automatic inspection manner for noise barrier using UAV images, and develops a fully convolutional network (FCN)-based noise barrier defect detection approach named skip connection YOLO detection network (SCYNet), which focuses on three aspects: network structure, loss function and data augmentation. First, a skip-connected feature structure Simi-BiFPN is incorporated into the network to fully fuse the features extracted from various scale layers without adding much computational overhead. Second, a NoiseIoU loss for bounding box regression is designed to improve existing IoU-based losses and get better performance on small dataset. Thirdly, a mixed sample data augmentation method named AutoFMix is proposed to eliminate the over-fitting issue caused by excessive similarity between samples, and further improve the detection accuracy. Finally, experiments conducted on the UAV railway noise barrier dataset show that the proposed SCYNet model achieves 92.2 mAP and 78.7 FPS, respectively, which outperform other models in terms of accuracy and processing speed. The fast-processing speed and high detection accuracy can quickly turn UAV images into useful information to assist railway maintenance, thereby improving the safety of train operation.
Regular inspections on track components such as the clip, spike, and rail are essential to maintain track quality and ensure railroad operating safety. Unfortunately, traditional image processing (IP)-based systems have limited accuracy. Existing convolutional neural network (CNN)-based approaches are designed for either detection or saliency segmentation of a specific track component (e.g. fastener or rail only). The overall track condition could not be evaluated because not all the track components are inspected simultaneously. This paper presents an all-in-one YOLO (AOYOLO) framework for multi-task track component inspection. First, a newly developed ConvNeXt-based backbone is constructed in AOYOLO to produce suitable hyperfeatures for both detection and segmentation tasks. Second, a novel U-shaped salient object segmentation branch is incorporated into AOYOLO to supplement the object detection branch, improving both the rail surface defect segmentation and the detection of other components. Advanced data augmentations are integrated to further enhance the accuracy and scalability of the network. Extensive experiments conducted on a track dataset established with images taken by drone indicate that the proposed system is able to: (1) achieve 95.6% mAP for track components inspection at a real-time speed of 147 fps, and (2) reach 93.6% accuracy on RSDs detection, which surpass the current state-of-the-art models. The exceptional inference speed and superior detection accuracy have great potential for field applications.
16 February 2024 in Advanced Engineering Informatics
Potential safety hazards (PSHs) around the high-speed railroad need to be detected and evaluated regularly and timely to ensure high-speed railroad operation safety. Unmanned aerial vehicle (UAV)-based PSH evaluation has great potential to supplement the current manual visual inspection tasks by providing better overhead views and less man-made accidents. This study presents an evaluation system for PSHs along high-speed railroad tracks. First, a novel hybrid learning architecture named UYOLO (U-shape You Only Look Once) is designed, which integrate the CSP-based backbone and detection branch to produce three scale high-level features for the object detection. Then, an innovative parsing branch inserted behind high-level layer of the structure progressively transmits context information to the shallow layer to accurately accomplish the pixel-level parsing task. Second, a new loss function using minimum point distance IoU (MPD-IoU) is designed and incorporated into the architecture to optimize the coordinate regression process of the predicted bounding boxes. Conveniently, an image-based hazard evaluation model is also developed and integrated to rate the hazard level of the detected PSHs. Finally, extensive experiments conducted on the track environment dataset established with UAV imagery indicate the proposed system can achieve a high detection rate yet remains efficient and convenient.