Object detection on railway tracks, which is crucial for train operational safety, face numerous challenges such as multiple types of objects and the complexity of train running environment. In this study, a multi-sensor framework is proposed to fuse camera and LiDAR data for the detection of objects on railway track including small obstacles and forward trains. The framework involves a two-stage process: region of interest extraction and object detection. In the first stage, a multi-scale prediction network is designed to achieve pixel level segmentation of the railway track and forward train via the image. In the second stage, LiDAR data is used to estimate the distance to the train and detect small obstacles in the railway track area which is extracted from the first stage. Experimental results show that the region of interest extraction method achieves desirable accuracy for railway track and train segmentation; and the proposed fusion method outperforms the one based on camera or LiDAR alone for small obstacles and forward train detection. Moreover, in practice the proposed framework has been successfully applied on the Hong Kong Metro TSUEN WAN line and the Beijing Metro YANFANG line.

Semantic segmentation is a crucial component of autonomous driving. However, the segmentation performance in unstructured roads is challenging owing to the following reasons: (1) irregular shapes and varying sizes of road boundaries, (2) low contrast or blurred boundaries between the road and background, and (3) environmental factors such as changing light intensities and dust particles. To overcome these challenges, this study proposes SwinURNet, a Transformer-convoluted neural network (CNN) architecture for real-time point cloud segmentation in unstructured scenarios. First, the point cloud is projected onto a range image via spherical projection. Then, a lightweight ResNet34-based network is designed for encoding abstract features. A non-square Swin transformer is designed for decoding information and capturing high-resolution transverse features. A multidimensional information fusion module is introduced to balance the semantic differences between feature maps in CNNs and attention maps in transformers. A multitask loss function comprising boundary, weighted cross-entropy, and Lovász–Softmax losses are introduced at the network end to guide network training. Experimental data from autonomous mining trucks in the Baiyuneboite mining area is used to evaluate the performance on unstructured roads. The proposed architecture is also applied to the public unstructured dataset RELLIS-3D and the large structured dataset SemanticKITTI. The experimental results show performance gains of 74.2, 42.6, and 61.6% in accuracy and 8–19 FPS inference speed with the proposed architecture, surpassing those of the compared methods.

Accurate train positioning is crucial for train safety. In this paper, we propose a train positioning method which fuses vison and millimeter-wave radar data. The proposed method contains two parts: loop closure detection (LCD) and radar-based odometry. The loop closure detection part fuses the convolutional neural network (CNN) features and the line features to achieve accurate key location detection. The radar-based odometry part proposes a train speed measurement algorithm using millimeter-wave radar, and combines the results of loop closure detection to further realize train positioning. Experiments conducted on the Hong Kong metro Tsuen Wan line show that our proposed loop closure detection can achieve an efficient key location detection with 98.57% precision and 99.37% recall; the speed detection method fulfills the ETCS requirements; and the relative error of the proposed train positioning method is 0.45%. Besides, the proposed method has been applied on the Hong Kong Metro TSUEN WAN line.

Rail track segmentation is key to environmental perception of autonomous train. However, due to the complexity of railway track environment, critical issues such as the detection of rail tracks with different curvatures remain to be overcome. In this study, a novel architecture called FarNet is proposed for long-range railway track point cloud segmentation. The proposed FarNet is mainly divided into three parts, i.e., spherical projection, attention-aggregation network and results refinement. Specifically, spherical projection converts the LiDAR point cloud into a pseudo range image, and attention-aggregation network enables railway track detection using the pseudo range image. Furthermore, in the attention-aggregation network two components, i.e., spatial attention module and information aggregation module, are proposed to enhance the capability of rail track segmentation. Last, the results refinement helps further filter out the noise points after segmentation. Experimental results show that the proposed FarNet achieved 98.0% mean intersection-over-union (MIoU) and 98.9% mean pixel accuracy (MPA) for rail track segmentation.

Roadside object detection and classification provide a good understanding of driving scenarios in regard to over-the-horizon perception. However, typical roadside sensors are insufficient when used separately. The fusion of the millimeter-wave (MMW) radar and monovision camera serves as an efficient approach. Unfortunately, the uncertain and conflicting data in extreme light conditions pose challenges to the fusion process. To this end, this study proposed an evidential framework to fuse the radar and camera data. A novel modeling approach for basic belief assignments (BBAs) was proposed, which took the uncertainty of convolutional neural network (CNN) model into consideration. Moreover, the single-scan and multiscan fusion methods were developed based on the enhanced evidence theory, which utilized different weighted coefficients by referring to the reinforced belief (RB) divergence measure and belief entropy (BE). Both numerical and empirical experiments were conducted to investigate the method performance. Specifically, in numerical experiments, the belief value of actual classification increased to 99.01%. For empirical experiments, based on the real datasets collected by roadside devices, the proposed method was demonstrated to outperform the state-of-the-art ones in terms of 71.06% and 87.23% precisions for bright light and low illumination conditions, respectively. The results verify that the proposed method is effective in fusing the conflicting and uncertain data.

Rail-track detection is a core function for automated rail transit perception. However, existing methods cannot effectively detect the rail-tracks in a complex environment, especially in turnout scenarios. In this study, we propose a topology guided method to detect rail-tracks which includes the following four parts: Firstly, a neural network is used to obtain the pixels of the rail-lanes, and the geometric relationship between rail-lanes is mined by inverse perspective transformation. Secondly, the rail-lanes’ pixels are converted to rail-lanes’ key points and the topological relationship between the key points. Thirdly, the rail-lanes are reconnected through topological relationships between key points. Finally, the rail-track geometry features are used to match the rail-lanes. Experimental results show that the rail-track level F1 score of the proposed method reached 91.62%, which is state-of-the-art (SOTA) in this field. Furthermore, the proposed method has been tested and applied on the Hong Kong Metro Tsuen Wan Line.