Three-dimensional feature descriptors are heavily employed in various 3-D perception applications to find keypoint correspondences between two point clouds. The availability of mobile devices equipped with depth sensors compels the developed applications to be both memory and computationally efficient. Toward this, in this letter, we present 3DHoPD, a new low-dimensional 3-D feature descriptor that is extremely fast to compute. The novelty lies in compactly encoding the “3-D” keypoint position by transforming it to a new 3-D space, where the keypoints arising from similar 3-D surface patches lie close to each other. Then, we propose histograms of point distributions (HoPD) to capture the neighborhood structure, thus forming 3DHoPD (3D+HoPD). We propose a tailored feature descriptor matching technique, wherein the “3-D” keypoint position in the new 3-D space is used to remove false positive matches, effectively reducing the search space by 90%, and then, the exact match is found using the “HoPD” descriptor. Experimental evaluation on multiple publicly available datasets shows that 3DHoPD is robust to noise and offers stable and competitive keypoint matching performance to the existing state-of-the-art 3-D descriptors with similar dimensionality across datasets, while requiring dramatically low-computational time (10× faster). The source code and additional experimental results are available at https://sites.google.com/site/3dhopd/.

Lidar-Visual-Inertial SLAM (LVINS) provides a compelling solution for accurate and robust state estimation and mapping, integrating complementary information from multi-sensor data. However, in the front-end processing of existing LVINS systems, methods based on visual line feature matching typically suffer from low accuracy and are time-consuming. Additionally, the back-end optimization of current multi-sensor fusion SLAM systems is adversely affected by feature association outliers, which constrains further enhancements in localization precision. In the loop closure process, existing lidar loop closure descriptors, relying primarily on 2D information from point clouds, often fall short in complex environments. To effectively tackle these challenges, we introduce the Multi-Modal Feature-based Lidar-Visual-Inertial SLAM framework, abbreviated as MMF-LVINS. Our framework consists of three major innovations. Firstly, we propose a novel coarse-to-fine visual line matching method that utilizes geometric descriptor similarity and optical flow verification, substantially improving both efficiency and accuracy of line feature matching. Secondly, we present a robust iterative optimization approach featuring a newly proposed adaptive loss function. This function is tailored based on the quality of feature association and incorporates graduated non-convexity, thereby reducing the impact of outliers on system accuracy. Thirdly, to augment the precision of lidar-based loop closure detection, we introduce an innovative 3D lidar descriptor that captures spatial, height, and intensity information from the point cloud. We also propose a two-stage place recognition module that synergistically combines both visual and this new lidar descriptor, significantly diminishing cumulative drift. Extensive experimental evaluations on six real-world datasets, including EuRoc, KITTI, NCLT, M2DGR, UrbanNav and UrbanLoco, demonstrate that our MMF-LVINS system achieves superior state estimation accuracy compared to existing state-of-the-art methods. These experiments also validate the effectiveness of our advanced techniques in visual line matching, robust iterative optimization, and enhanced lidar loop closure detection.

We propose a lifelong 3D mapping framework that is modular, cloud-native by design and more importantly, works for both hand-held and robot-mounted 3D LiDAR mapping systems. Our proposed framework comprises of dynamic point removal, multi-session map alignment, map change detection and map version control. First, our sensor-setup agnostic dynamic point removal algorithm works seamlessly with both hand-held and robot-mounted setups to produce clean static 3D maps. Second, the multi-session map alignment aligns these clean static maps automatically, without manual parameter fine-tuning, into a single reference frame, using a two stage approach based on feature descriptor matching and fine registration. Third, our novel map change detection identifies positive and negative changes between two aligned maps. Finally, the map version control maintains a single base map that represents the current state of the environment, and stores the detected positive and negative changes, and boundary information. Our unique map version control system can reconstruct any of the previous clean session maps and allows users to query changes between any two random mapping sessions, all without storing any input raw session maps, making it very unique. Extensive experiments are performed using hand-held commercial LiDAR mapping devices and open-source robot-mounted LiDAR SLAM algorithms to evaluate each module and the whole 3D lifelong mapping framework.

Global visual localization estimates the absolute pose of a camera using a single image, in a previously mapped area. Obtaining the pose from a single image enables many robotics and augmented/virtual reality applications. Inspired by latest advances in deep learning, many existing approaches directly learn and regress 6 DoF pose from an input image. However, these methods do not fully utilize the underlying scene geometry for pose regression. The challenge in monocular relocalization is the minimal availability of supervised training data, which is just the corresponding 6 DoF poses of the images. In this letter, we propose to utilize these minimal available labels (i.e., poses) to learn the underlying 3D geometry of the scene and use the geometry to estimate the 6 DoF camera pose. We present a learning method that uses these pose labels and rigid alignment to learn two 3D geometric representations ( X, Y, Z coordinates ) of the scene, one in camera coordinate frame and the other in global coordinate frame. Given a single image, it estimates these two 3D scene representations, which are then aligned to estimate a pose that matches the pose label. This formulation allows for the active inclusion of additional learning constraints to minimize 3D alignment errors between the two 3D scene representations, and 2D re-projection errors between the 3D global scene representation and 2D image pixels, resulting in improved localization accuracy. During inference, our model estimates the 3D scene geometry in camera and global frames and aligns them rigidly to obtain pose in real-time. We evaluate our work on three common visual localization datasets, conduct ablation studies, and show that our method exceeds state-of-the-art regression methods' pose accuracy on all datasets.