The precise localization of rail vehicles is crucial for their secure operation. In this letter, we present a visual landmark-assisted light detection and ranging (LiDAR) and inertial fusion localization scheme, which adeptly addressing the key issues of absent global navigation satellite system (GNSS) signal and loopback-free areas encountered in railroad. To obtain a reliable visual landmark, we design a novel easy-to-deploy trackside kilometer post. Advanced deep learning based programs are employed for its identification and character detection. View cone and deskewed point cloud plane are combined to achieve accurate positioning of the kilometer post’s vertices. Further, we construct a positioning framework based on fast direct LiDAR-inertial odometry (FAST-LIO2) that adapts to the train operating environment. This framework employs an error-state extended kalman filter (ESKF) to fuse LiDAR and inertial data as a frontend, constructs a factor graph for optimizing the multimodal information to obtain the best position estimation at the backend. It’s worth noting that we leverage kilometer post plane factor and virtual vertex factor to carry out trajectory correction. Finally, our practical experiments demonstrate the excellent performance of the proposed method, achieving meter-level localization errors over long sequence of 8km. Kilometer post identification and digit extraction excel in a variety of complex situations. Thanks to the auxiliary role of the kilometer post, it still provides stable and accurate position estimation even in severely degraded tunnel scenarios. And it can meet the real-time localization requirements of trains.
To overcome the limitations of current train positioning systems, including low positioning accuracy and heavy reliance on track transponders or GNSS signals, this paper proposes a novel LiDAR-inertial and visual landmark fusion framework. Firstly, an IMU preintegration factor considering the Earth’s rotation and a LiDAR-inertial odometry factor accounting for degenerate states are constructed to adapt to railway train operating environments. Subsequently, a lightweight network based on YOLO improvement is used for recognizing reflective kilometer posts, while PaddleOCR extracts numerical codes. High-precision vertex coordinates of kilometer posts are obtained by jointly using LiDAR point cloud and an image detection box. Next, a kilometer post factor is constructed, and multi-source information is optimized within a factor graph framework. Finally, onboard experiments conducted on real railway vehicles demonstrate high-precision landmark detection at 35 FPS with 94.8% average precision. The proposed method delivers robust positioning within 5 m RMSE accuracy for high-speed, long-distance train travel, establishing a novel framework for intelligent railway development.
Low-altitude remote sensing technology using unmanned aerial vehicles (UAVs) has gained wide application. However, compared with satellites, UAVs have a smaller field of view and require image-stitching techniques. Existing stitching algorithms are generally only effective when processing images collected by drones at fixed positions but have poor performance on images collected by drones at different locations. This poses challenges in stitching due to large parallax caused by the movement of foreground objects, resulting in issues such as ghosts and misalignment. To address this, an algorithm based on deep features and optimal seams is proposed for large-parallax UAV remote sensing image stitching. In image registration, the algorithm uses LightGlue to implement feature point extraction and matching. On the basis of accurate image registration, image pre-alignment is achieved using a homography matrix. We construct an energy function using color and structural features and then use dynamic programming to search for the optimal seam in the overlapping area that minimizes the energy function. Finally, we achieve stitching based on the optimal seam. In terms of evaluation metrics, in addition to peak signal-to-noise ratio and structural similarity index measure, we introduce an evaluation metric based on pixel patches for seams, making the quantitative evaluation more objective.