12 June 2024 in IEEE Transactions on Instrumentation and Measurement
Semantic segmentation in railway scenes constitutes a fundamental task in the context of autonomous train driving, serving as a critical source of positional information for other visual tasks. Yet, prior semantic segmentation algorithms have faced challenges in achieving real-time and precise railway panoptic segmentation. Railway scene images often have highly imbalanced content density. Hence, the distribution of different regions in the image is highly non-uniform, which leads to the issue of semantic sample imbalance. To achieve this, we propose a dense multi-scale fusion network (DMFNet), a lightweight and precise semantic segmentation framework tailored for railway scenes. DMFNet employs a network framework that combines dense connections and multi-scale fusion to efficiently extract and utilize features. To enhance the feature extraction capability of the neural network, we design a lightweight residual block with multiple scales, composed of large convolution kernels. Additionally, we introduce a novel method for reassigning loss function weights to balance the samples within railway scenes. Extensive experimentation demonstrates that our DMFNet achieves outstanding results on railway scene datasets. It is designed to provide a lightweight and accurate semantic segmentation solution for railway scenes.
26 February 2025 in IEEE Transactions on Industrial Informatics
Point cloud-based 3D object detection technology provides precise information for detecting obstacles in front of trains. In rail transit scenarios, obstacles are usually located at a distance, and the sparsity of point cloud data leads to information loss during feature extraction and spatial transformation, adversely affecting the accuracy of obstacle detection. To tackle this issue, we propose RAE3D, a single-stage end-to-end architecture that incorporates a multiscale voxel feature aggregation module on point queries (MVA-PQ) and a bird's-eye view multilevel auxiliary (BEV-MLA) module for efficient 3D object detection. The MVA-PQ module encodes multiscale voxel features into centroids of the final voxel layer, integrating spatial and scale information. The BEV-MLA module converts shallow 3D features into sparse 2D features using height compression, guiding the network to learn the spatial structure of small objects. We also introduce a 3D centroid offset loss for optimizing bounding boxes. Extensive experiments on the Rail3D and KITTI datasets have demonstrated the superiority of the proposed RAE3D.
21 June 2024 in IEEE Transactions on Instrumentation and Measurement
Anomaly detection of the railway track is essential to protect the safety of railway transportation. However, the railway track is often disturbed by unknown anomalies, making it challenging to detect anomalies using only supervised learning and visual sensors. To address these problems, this paper proposes a railway track anomaly detection method using semi-supervised learning and vision-lidar decision fusion. The method consists of three parts: supervised pre-detection, semi-supervised re-detection, and decision-level fusion. In the pre-detection stage, an enhanced instance segmentation algorithm is used to predetect known and unknown anomalies to improve the overall performance. In the re-detection stage, the reconstruction method, semi-supervised learning, and structured light point cloud are combined to re-detect anomalies, especially unknown anomalies. In the fusion stage, the fusion of pre-detection and redetection at the decision level further advances the accuracy and credibility of anomaly detection. Experimental results show that the proposed method achieves a mean average precision (mAP 0.50:0.95) of 92.2% and a mean average recall (mAR 0.50:0.95) of 94.5%. The proposed method is superior to the single vision-based or point cloud-based methods in anomaly detection. In addition, the proposed method provides a general solution to anomaly detection and extends the application of structured light point cloud.
23 December 2024 in IEEE Intelligent Transportation Systems Magazine
Accurate and reliable train positioning is essential for safe railway operations. However, traditional infrastructure-based methods lack the ability for autonomy and continuous positioning. To solve the problem, we propose a novel modular visual processing framework for continuous train positioning. Environmental information captured by stereo vision sensors is processed using a feature extraction algorithm to detect landmarks along the railway. The detection results are input into a dual-stream parallel processing architecture, where an anchor-based stereo matching module calculates the depth values of detected landmarks, providing essential spatial information. Simultaneously, a dynamic region of interest-based multiobject tracking module assigns unique identification codes to each landmark, facilitating the integration of landmark detection results with their corresponding depth values. This correlation allows for the tracking of changes in landmark positions and the calculation of displacement caused by train motion, leading to a more accurate estimation of the train's location. Subsequently, the train mileage is matched against a database track map to obtain the final precise position. Field tests on Beijing Metro Line 9 and the Capital Airport Line demonstrate an average mean relative error of 0.19% and positioning accuracy within 1 m over a range of 100 m.
23 August 2025 in Pattern Recognition