At present, the method of two-dimensional image recognition is mainly used to detect the abnormal fastener in the rail-track inspection system. However, the too-tight-or-too-loose fastener condition may cause the clip of the fastener to break or loose due to the high frequency vibration shock, which is difficult to detect from the two-dimensional image. In this practical application background, 3D visual detection technology provides a feasible solution. In this paper, we propose a fundamental multi-source visual data detection method, as well as an accurate and robust fastener location and nut or bolt segmentation algorithm. By combining two-dimensional intensity information and three-dimensional depth information generated by the projection of line structural light, the locating of nut or bolt position and accurate perception of height information can be realized in the dynamic running environment of railway. The experimental results show that the static measurement accuracy in the vertical direction using the structural light vision sensor is 0.1 mm under the laboratory condition, and the dynamic measurement accuracy is 0.5 mm under the dynamic train running environment. We use dynamic template matching algorithm to locate fasteners from 2D intensity map, which achieves 99.4% accuracy, then use the watershed algorithm to segment the nut and bolt from the corresponding depth image of located fastener. Finally, the 3D shape of the nut and bolt is analyzed to determine whether the nut or bolt height meets the local statistical threshold requirements, so as to detect the hidden danger of railway transportation caused by too loose or too tight fasteners.

Railway plug defects impact the safety of a railway system. To detect railway plug defects, we establish the framework of a visual inspection system (VIS), which is the first system that can perform railway plug inspection automatically and intelligently. Using the idea of change detection, the framework includes three algorithm modules, which are named the object location, image alignment and similarity measurement modules. After the image acquisition system captures a rail image as the input, the three algorithm modules process the image in order. First, in the object location module, a deep convolutional neural network is used to perform plug location. Second, in the image alignment module, a simple and fast method is designed to align key images using histogram of oriented gradients features. Third, in the similarity measurement module, the X2 distance is used to compute the similarity between the two plug regions in an inspection image and in an aligned ground-truth image. The results of the similarity measurement are sorted when all inspection images are processed. Therefore, the inspection images with smaller similarity values are ranked higher and the plugs in the images have larger probabilities of defects. The framework has passed the practice tests, and the visual inspection system using this framework has already been authorized by the China Railway Corporation and will be equipped in many inspection trains belonging to local railway corporations.

The problem of foreign object intrusion onto the track bed often occurs in the actual operation process of high-speed railways. To solve the problem, we propose an anomaly detection method for the ballastless track bed, which is based on semantic segmentation. Firstly, we put forward the RFODLab semantic segmentation network according to the randomness of foreign objects distribution, and a small proportion of target pixels in the track image. The segmentation results of track image obtained through this model can be used to obtain the accurate pixel information of foreign objects. To further improve the recall and precision, the channel attention mechanism is introduced for the backbone network of the model to aggregate the context information of images, which achieves the weighted constraints of the model on the area to be recognized. Furthermore, to improve the model performance affected by unbalanced sample category distribution during the anomaly detection, we modify the loss function by balancing distribution of each category. The experimental results show that our proposed method can effectively segment various types of anomalies on the ballastless track bed including broken elastic strips, animal carcasses, and fallen pieces. The precision of anomaly detection on the test set can reach 90% while the recall can be maintained at more than 95%. The anomaly detection results on actual lines also verify the effectiveness of the method.

The contact wire wear is an important parameter to ensure the safety operation of electric railways. The contact wire may break if the wear is serious, which leads to transportation interruptions. This study proposes an optical measurement method of contact wire wear, using stereovision technology. The matching method of stereovision based on line-scan cameras is proposed. A lookup-table method is developed to exactly determine the image resolution caused by the contact wire being in different spatial positions. The wear width of the contact wire is extracted from catenaries’ images, and the residual thickness of the contact wire is calculated. The method was verified by field tests. The round-robin tests of the residual thickness at the same location present excellent measurement repetitiveness. The maximum difference value between dynamic test results and ground measurement results is 0.13 mm. This research represents a potential way to implement condition-based maintenance for contact wire wear in the future in order to improve the maintenance efficiency and ensure the safety of catenary infrastructure.

Trackside communication and signal infrastructure (TCSI) plays an important role in the safe operation of a high-speed railway system. However, it is a challenge to inspect such infrastructure using a vision-based system because of the uneven illumination and various reflection properties of the TCSI surfaces. This paper puts forward a visual inspection system installed on an inspection car for TCSI. The visual inspection system first captures the trackside images using the image acquisition system. Then, an image processing method using a varying-scale Retinex algorithm is implemented so that the visual inspection system can enhance the contrast of these trackside images. The method changes the size of the Gaussian kernel adaptively according to the image details in order to prevent halo artifacts and graying-out effects inherent in many image enhancement algorithms. By comparison with several state of the art methods, varying-scale Retinex is validated to overcome the local inhomogeneous brightness of images and make those images more suitable for browsing and analysis for inspectors. Composed of the image acquisition system and varying-scale Retinex, the visual inspection system is installed on many inspection cars in China for performing visual inspection.

Brake rotors are critical parts of the disc braking system for modern vehicles. One common failure for brake rotors is the thickness variation, which may result in unpleasant brake pulsation, vehicle vibration during braking, or eventually lead to the malfunction of the braking system. In order to improve customer satisfaction, vehicle serviceability and availability, it is necessary to develop an onboard fault detection and isolation solution. In our previous work, the vibration features of master cylinder pressure, vehicle longitudinal acceleration and wheel speed were identified as fault signatures. Based on these fault signatures, a vibration- based fault detection and isolation algorithm is developed in this work. The difference of frequency response between the braking period and the normal driving period (non-braking) is employed to improve the algorithm robustness. The experiment results demonstrate the proposed algorithm can robustly diagnose the thickness variation fault and isolate the fault to each vehicle corner.