13 December 2023 in IEEE Robotics and Automation Letters
Recently, cross-source point cloud registration from different sensors has become a significant research focus. Although current methods have advanced homogenous point cloud registration, challenges persist in the cross-source domain due to varying point cloud densities from different sensors and missing points caused by different viewing angles, which have hindered its development. To address these issues, we propose HybridFusion, a novel algorithm designed specifically for cross-source point cloud registration in outdoor large-scale scenes, accommodating various sensors and viewing angles. Due to the unique characteristics of point clouds, it is not a singular module, but rather a coarse-to-fine process. To extract similarity information from cross-source point clouds, local patches of the point cloud are subjected to similarity matching. Subsequently, precise alignment is performed using their distinctive features, including 2D boundary points. Finally, the poses obtained from multiple patches are fused to achieve the final registration. Our proposed approach is extensively evaluated through both qualitative and quantitative experiments with existing methods. Additionally, a novel metric for point cloud completion is introduced. The results establish our method as the state-of-the-art solution for cross-source point cloud registration, with a remarkable 70% increase in accuracy compared to recent approaches.
13 February 2025 in IEEE Transactions on Automation Science and Engineering
In this paper, we introduce G2-Mapping, a novel method to comprehensively support online monocular, RGB-D, and LiDAR-Inertial-Visual systems, employing 3D gaussian points as scene representation. There are several issues when applying 3d gaussian splatting (3DGS) techniques to simultaneous localization and mapping (SLAM) 1) for monocular, the lack of depth information makes scene initialization difficult and large baseline positioning challenging; 2) differentiable rendering with respect to depth and pose has not been implemented in 3DGS, making it difficult to directly apply to the SLAM system; 3) strategy for updating the scene with incoming online frames is not present, which may lead to memory overflow. In order to overcome problems mentioned above, we formulate a mathematical derivation and propose a differentiable rendering approach that leverages both depth and color to optimize the scene and pose. We introduce a simplified odometry that provides a metric depth estimation for monocular and enhance the low-overlap scene availability. A scale consistency and uncertainty weighted optimization is further proposed to eliminates the impact of inaccurate depth prediction. Our proposed scene updating strategy effectively prevents rapid memory growth. Tracking and mapping are performed alternatively to achieve precise localization and synchronous high-fidelity map reconstruction. Extensive experiments demonstrate that our G2-Mapping surpasses feature-based SLAM in localization precision and exceeds state-of-the-art neural SLAM methods in the fidelity of view synthesis. Note to Practitioners—This paper is dedicated to tackling the efficiency challenges in multi-source SLAM and map reconstruction, aiming to generate pose and high-fidelity maps synchronously. We introduce G2-Mapping, a novel framework that leverages the power of 3D Gaussian points for scene representation, offering a universal solution for monocular, RGB-D, and LiDAR-Inertial-Visual systems. By developing a comprehensive differentiable renderer and presenting a strategy for dynamic scene updating, G2-Mapping significantly advances the state-of-the-art in localization precision and view synthesis fidelity. Although the approach is highly promising, it currently relies on the accuracy of depth prediction networks and requires further optimization for handling sparse LiDAR data. Future research will focus on enhancing these aspects, aiming to seamlessly integrate G2-Mapping into practical applications within robotics, autonomous vehicles, and augmented reality, where robust and efficient SLAM solutions are paramount.
19 January 2026 in IEEE Robotics and Automation Letters
We propose GauS-SLAM, a dense RGB-D SLAM system that leverages 2D Gaussian surfels to achieve robust tracking and high-fidelity mapping. Our investigations reveal that Gaussian-based scene representations exhibit geometry distortion under novel viewpoints, which significantly degrades the accuracy of Gaussian-based tracking methods. These geometry inconsistencies arise primarily from the depth modeling of Gaussian primitives and the mutual interference between surfaces during the depth blending. To address these, we propose a 2D Gaussian-based incremental reconstruction strategy coupled with a Surface-aware Depth Rendering mechanism, which significantly enhances geometry accuracy and multi-view consistency. Additionally, the proposed local map design dynamically isolates visible surfaces during tracking, mitigating misalignment caused by occluded regions in global maps while maintaining computational efficiency with increasing Gaussian density. Extensive experiments across multiple datasets demonstrate that GauS-SLAM outperforms comparable methods, delivering superior tracking precision and rendering fidelity.
16 June 2025 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Cross-view image-based geolocalization can provide accurate, drift-free navigation in the absence of external positioning signals, which is particularly crucial in domains such as UAV delivery logistics and disaster relief. While existing research predominantly focuses on matching ground panoramic images with known orientations, real-world scenarios often present images with unknown orientations. Additionally, acquiring panoramic images with a full $360^\circ$ field of view (FoV) is impractical, creating a disconnect between research and practical applications. In this study, we introduce an innovative cross-view geolocalization method that integrates sophisticated orientation estimation. Our goal is to improve geolocalization accuracy in limited FoV and arbitrary orientation scenarios. To achieve this, we design an efficient hard sample training strategy employing a dual-weighted soft-margin triplet loss function to extract discriminative features. Concurrently, we develop an orientation estimation module that leverages a convolution-based sliding window to assess the similarity with satellite-view embeddings and query embeddings. Thus, our method is named OriLoc. We validated the approach's unique outperformance on three challenging cross-view geolocalization datasets, including data from commercial, residential, urban landscapes, and suburban areas across two continents, aligning with the demands of real-world scenarios. Our results demonstrate that a straightforward hard sample mining strategy, when combined with an appropriate learning objective, significantly enhances geolocalization performance for limited FoV and orientation-free cross-view images, as shown in Figure 1. Furthermore, our orientation estimation module, which incorporates classical convolutions, exhibits remarkable accuracy in estimating orientations when integrated with attention embeddings prior to polar transformation. Code and trained models are publicly available on https://github.com/boni-hu/OriLoc.
21 March 2024 in IEEE Transactions on Geoscience and Remote Sensing
Visual geolocalization is a cost-effective and scalable task that involves matching one or more query images, taken at some unknown location, to a set of geotagged reference images. Existing methods, devoted to semantic features representation, evolving towards robustness to a wide variety between query and reference, including illumination and viewpoint changes, as well as scale and seasonal variations. However, practical visual geolocalization approaches need to be robust in appearance changing and extreme viewpoint variation conditions, while providing accurate global location estimates. Therefore, inspired by curriculum design, human learn general knowledge first and then delve into professional expertise. We first recognize semantic scene and then measure geometric structure. Our approach, termed CurriculumLoc , involves a delicate design of multi-stage refinement pipeline and a novel keypoint detection and description with global semantic awareness and local geometric verification. We rerank candidates and solve a particular cross-domain perspective-n-point (PnP) problem based on these keypoints and corresponding descriptors, position refinement occurs incrementally. The extensive experimental results on our collected dataset, TerraTrack and a benchmark dataset, ALTO , demonstrate that our approach results in the aforementioned desirable characteristics of a practical visual geolocalization solution. Additionally, we achieve new high recall@1 scores of 62.6% and 94.5% on ALTO, with two different distances metrics, respectively. Dataset, code and trained models are publicly available on https://github.com/npupilab/CurriculumLoc.