The condition monitoring of railway track line is one of the essential tasks to ensure the safety of the railway transportation system. Railway track line is mainly composed of tracks, fasteners, sleepers, and so on. Given the requirements for rapid and accurate inspection, innovative and intelligent methods for multi-target defect identification of the railway track line using image processing and deep learning methods are proposed in this paper. Firstly, the track and fastener positioning method based on variance projection and wavelet transform is introduced. After that, a bag-of-visual-word (BOVW) model combined with spatial pyramid decomposition is proposed for railway track line multi-target defect detection with a detection accuracy of 96.26%. Secondly, an improved YOLOv3 model named TLMDDNet (Track Line Multi-target Defect Detection Network), integrating scale reduction and feature concatenation, is proposed to enhance detection accuracy and efficiency. Finally, to reduce model complexity and further improve the detection speed, with the help of dense connection structure, a lightweight design strategy for the TLMDDNet model named DC-TLMDDNet (Dense Connection Based TLMDDNet) is proposed, in which the DenseNet is applied to optimize feature extraction layers in the backbone network of TLMDDNet. The effectiveness of the proposed methods is demonstrated by the experimental results.

Railway intrusion seriously threatens railway safety and can cause enormous loss of life and property. Therefore, railway intrusion detection is crucial for the safety of railway operation. Among the current methods of intrusion detection, machine vision-based methods have been widely used in railways, and have attracted close attention because of their great benefits. This paper proposes a comprehensive review of railway intrusion detection based on machine vision, covering ground monitoring, on-board inspection, and unmanned aerial vehicle (UAV) inspection. First, this paper systematically reviews most of the studies over the past two decades and presents the survey in three parts. Second, by analyzing these studies and the requirements for railway monitoring, we summarize the major challenges that hinder railway intrusion detection based on machine vision. Finally, we propose several promising perspectives for railway intrusion detection based on machine vision by comprehensively considering the development of machine vision, sensors, and pattern recognition together with the needs of railway scenes.

Pantograph is one of the most important components in electrical railway vehicles. To guarantee steady power supply for the train, the surface of the pantograph slide plate should be smooth enough so that the catenary can move on it from one side to the other side steadily with low friction. In addition, the thickness of the pantograph slide plate cannot be smaller than the lower limit for the sake of safety. Therefore, periodical inspection and maintenance of the pantograph slide plate are significant in terms of safe and stable operation. In this paper, an innovative and intelligent method based on deep learning and image processing technologies is proposed for the online condition monitoring of the pantograph slide plate. In the first stage, the surface defect detection and recognition method of the pantograph slide plate is proposed. Four typical surface defects of the slide are considered, and a deep learning model, pantograph defect detection neural network (PDDNet), is trained for the defect detection and recognition. In the second stage, five key criteria for qualifying the wear condition are proposed. The wear edge estimation based on image processing technology is investigated in detail. Furthermore, they are used to calculate the wear depth and evaluate the wear condition of the pantograph slide. The experiment results demonstrate that the proposed PDDNet can detect the surface defects and also recognize the four kinds of defects with a sound accuracy. The wear depth estimation results are compared with on-site measurement data, and the proposed method can achieve high estimation accuracy.

Fault diagnosis of wheelset bearings in high-speed trains has attracted constant interest in the scientific community and industrial field. Under the harsh working condition, e.g., time-varying speed and load, most existing methods are hindered by the limited and unknown situations of wheelset bearings. Although the self-calibrated convolution is proven to effectively expand the receptive field with more accurate discriminative regions, its use in fault diagnosis still lacks needed physical interpretation as well as computational efficiency. To this end, this article presents a novel framework by using the logarithmic short-time Fourier transform and the modified self-calibrated convolution. It first manifests a time-frequency map that has explicit physics meaning while reducing the gap between high energy and detailed characteristics in the masking of interfering signals. To simplify redundant kernels, a modified self-calibrated residual block is proposed without introducing any more parameters, while preserving an interpretable and simple structure. The effectiveness and robustness of the proposed method are verified by the experimental data collected from an industrial railway axle bearing test rig. Results are found superior to those of five state-of-art methods, which are more practical in terms of accuracy, cost time, and model size.

Rail surface defects seriously affect the safety of railway systems. At present, human inspection and rail vehicle inspection are the main approaches for the detection of rail surface defects. However, there are many shortcomings to these approaches, such as low efficiency, high cost, and so on. This paper presents a novel visual inspection approach based on unmanned aerial vehicle (UAV) images, and focuses on two key issues of UAV-based rail images: image enhancement and defects segmentation. With regards to the first aspect, a novel image enhancement algorithm named Local Weber-like Contrast (LWLC) is proposed to enhance rail images. The rail surface defects and backgrounds can be highlighted and homogenized under various sunlight intensity by LWLC, due to its illuminance independent, local nonlinear and other advantages. With regards to the second, a new threshold segmentation method named gray stretch maximum entropy (GSME) is presented in this paper. The proposed GSME method emphasizes gray stretch and de-noising on UAV-based rail images, and selects an optimal segmentation threshold for defects detection. Two visual comparison experiments were carried out to demonstrate the efficiency of the proposed methods. Finally, a quantitative comparison experiment shows the LWLC-GSME model achieves a recall of 93.75% for T-I defects and of 94.26% for T-II defects. Therefore, LWLC for image enhancement, in conjunction with GSME for defects segmentation, is efficient and feasible for the detection of rail surface defects based on UAV Images.

The operation of trains is substantially jeopardized by the presence of railway obstacles. Recent research on vision-based railway obstacle detectors has emphasized using sizeable neural network models to pursue a high accuracy while neglecting power consumption and training costs. To accelerate the model updating, training, and deployment cycle applied in the railway safety guarantee system, this article proposes a lightweight three-stage detection framework to identify obstacles in the single railway image. It is mainly comprised of the coarse region proposal (CRP) module, the lightweight railway obstacle detection network (RODNet), and postprocessing stage. In the CRP module, we set the binarized normed gradient (BING) detector to work on two different scale images for a more effective generation of proposal regions. Then, the intersection-over-minimum (IoM) metric is presented to filter and merge generated subregions. In the RODNet, a more lightweight and effective detection model is designed based on substantial improvements of YOLOv4-tiny, to simultaneously predict bounding boxes on the whole image and in each subregion. The RODNet’s backbone is rebuilt with one-shot aggregation and attention mechanism to reduce parameters and to maintain feature extraction abilities. Moreover, some residual blocks are also integrated into RODNet’s neck and head to improve the detection performance. Finally, a series of operations is devised to fuse, remove, and suppress predicted boxes on the boundary of each candidate subregion to reduce false-positive results at the postprocessing stage. Experimental results in the railway obstacle dataset show that RODNet reduces 48% parameters and 29% floating-point operations per second (FLOPs) with at least a 68.2% improvement of inference speed in the CPU compared with YOLOv4-tiny. Meanwhile, the proposed framework provides 80.3 mean average precision (mAP) at a speed of 65.9 ms/pic in the low-power graphics processing unit (GPU) and achieves a competitive or even better performance compared with the state-of-the-art large detection models.