Global localization in 3D point clouds is a challenging task for mobile vehicles in outdoor scenarios, which requires the vehicle to localize itself correctly in a given map without prior knowledge of its pose. This is a critical component of autonomous vehicles or robots on the road for handling localization failures. In this paper, based on reduced dimension scan representations learned from neural networks, a solution to global localization is proposed by achieving place recognition first and then metric pose estimation in the global prior map. Specifically, we present a semi-handcrafted feature learning method for 3D Light detection and ranging (LiDAR) point clouds using artificial statistics and siamese network, which transforms the place recognition problem into a similarity modeling problem. Additionally, the sensor data using dimension reduced representations require less storage space and make the searching easier. With the learned representations by networks and the global poses, a prior map is built and used in the localization framework. In the localization step, position only observations obtained by place recognition are used in a particle filter algorithm to achieve precise pose estimation. To demonstrate the effectiveness of our place recognition and localization approach, KITTI benchmark and our multi-session datasets are employed for comparison with other geometric-based algorithms. The results show that our system can achieve both high accuracy and efficiency for long-term autonomy.
3 January 2020 in IEEE Transactions on Intelligent Transportation Systems
Large scale 3D maps constructed via LiDAR sensor are widely used on intelligent vehicles for localization in outdoor scenes. However, loading, communication and processing of the original dense maps are time consuming for onboard computing platform, which calls for a more concise representation of maps to reduce the complexity but keep the performance of localization. In this paper, we propose a teacher-student learning paradigm to compress the 3D point cloud map. Specifically, we first find a subset of LiDAR points with high number of observations to preserve the localization performance, which is regarded as the teacher of map compression. An efficient optimization strategy is proposed to deal with the massive data in original map. With the supervision of compressed map, a student model is built by training a random forest model fed with geometric feature descriptors of each point. As a result, the student model is able to compress the map without referring to the expensive numerical optimization. Additionally, by incorporating the features, the innovative student model can be generalized to other new maps while no re-training is required. We conduct thorough experiments on multi-session dataset and KITTI dataset to demonstrate the effectiveness and efficiency of the proposed learning paradigm, and the comparison with other map compression methods. The final results show that the learned student model can achieve efficient map compression with comparable LiDAR based localization performance to the original map at the same time.
14 January 2021 in IEEE Transactions on Instrumentation and Measurement
This article presents a high-precision single-camera inertial measurement unit (IMU) extrinsic calibration method by tightly fusing the visual information from other cameras. Specifically, multiple additional cameras are added to the monocular camera-IMU system for assisting calibration as we theoretically prove that more cameras used in calibration can lead to smaller lower bound on the covariance of the estimated extrinsic parameters, which then results in better calibration accuracy. Moreover, we provide two degenerative motion conditions in the resulting multicamera visual-inertial system, which impair the calibration accuracy and should be avoided in real application whenever possible. More importantly, we present the requirement of minimum motion for a reliable extrinsic calibration to provide the practical guideline. Finally, the full validation on both simulation and real-world data is demonstrated. By evaluating the Cramér-Rao lower bound on the covariance, the proposed camera-IMU calibration method is shown to be statistically efficient for accurate calibration with errors less than 0.01 m in translation and 0.5° in rotation, which is consistent with the theoretical analysis in this article.
Accurate LiDAR-camera extrinsic calibration is a precondition for many multi-sensor systems in mobile robots. Most calibration methods rely on laborious manual operations and calibration targets. While working online, the calibration methods should be able to extract information from the environment to construct the cross-modal data association. Convolutional neural networks (CNNs) have powerful feature extraction ability and have been used for calibration. However, most of the past methods solve the extrinsic as a regression task, without considering the geometric constraints involved. In this paper, we propose a novel end-to-end extrinsic calibration method named DXQ-Net, using a differentiable pose estimation module for generalization. We formulate a probabilistic model for LiDAR-camera calibration flow, yielding a prediction of uncertainty to measure the quality of LiDAR-camera data association. Testing experiments illustrate that our method achieves a competitive with other methods for the translation component and state-of-the-art performance for the rotation component. Generalization experiments illustrate that the generalization performance of our method is significantly better than other deep learning-based methods.
14 December 2020 in IEEE Transactions on Instrumentation and Measurement
Online dense mapping gives a representation of the unstructured terrain, which is indispensable for safe robotic motion planning. In this paper, we propose such an elevation mapping system, namely GEM, to generate a dense local elevation map in constant real-time for fast responsive local planning, and maintain a globally consistent dense map for path routing at the same time. We model the global elevation map as a collection of submaps. When the trajectory estimation of the robot is corrected by SLAM, only relative poses between submaps are updated without re-building the submap. As a result, this deformable global dense map representation is able to keep the global consistency online. Besides, we accelerate the local mapping by integrating traversability analysis into the mapping system to save the computation cost by obstacle awareness. The system is implemented by CPU-GPU coordinated processing to guarantee constant real-time performance for in-time handling of dynamic obstacles. Substantial experimental results on both simulated and real-world dataset validate the efficiency and effectiveness of GEM.
7 April 2020 in IEEE Transactions on Industrial Electronics
Robust and efficient visual localization is essential for numerous robotic applications. However, it remains a challenging problem especially when significant environmental or perspective changes present, as there are high percentage of outliers, i.e. incorrect feature matches, between the query image and the map. In this paper, we propose a novel 2-entity RANSAC framework using 3D-2D point and line feature matches for visual localization with the aid of inertial measurements and derive minimal closed form solutions using only 1 point 1 line or 2 point matches for both monocular and multi-camera system. The proposed 2-entity RANSAC can achieve higher robustness against outliers as multiple types of features are utilized and the number of matches needed to compute a pose is reduced. Furthermore, we propose a learning-based sampling strategy selection mechanism and a feature scoring network to be adaptive to different environmental characteristics such as structured and unstructured. Finally, both simulation and real-world experiments are performed to validate the robustness and effectiveness of the proposed method in scenarios with long-term and perspective changes $^{1}$ .