24 June 2024 in IEEE Transactions on Instrumentation and Measurement
The autonomous flight of Unmanned Aerial Vehicle (UAV) relies on precise and robust geo-localization system. Visual geo-localization system registers the aerial captured image to a geo-referenced satellite map, which enables UAVs to determine their global position without Global Navigation Satellite System (GNSS). However, it is challenging to achieve precise and robust geo-localization in large-scale environment due to notable texture difference between the satellite map and the UAV-captured image. In this paper, we design a robust visual geo-localization pipeline which integrates a proposed deep learning-based imagery feature. This pipeline starts with the image retrieval based on the deep feature encoding, to initialize the localization process over large-scale maps without any location prior. With the reuse of the same deep imagery feature, an image registration process enables the real-time sequential localization. Besides, the proposed system has the re-localization ability to eliminate the localization drift caused by the possible registration failure, especially during the long-time flight. Evaluations on datasets and real-world experiments demonstrate that the proposed system is more robust and accurate than other state-of-the-art methods. To encourage the further progress on visual geo-localization problem, our code and materials are publicly available at https://github.com/hmf21/ UAVLocalization.
9 August 2024 in IEEE Robotics and Automation Letters
Visual localization plays an essential role in the autonomous flight of Unmanned Aerial Vehicles (UAVs) especially for the Global Navigation Satellite System (GNSS) denied environments. Existing aerial-based visual localization methods mainly focus on eliminating image variance between database map and captured frames. However, these is a lack of public dataset and baseline for method comparisons, which impedes the development of aerial-based visual localization. To address this issue, we construct AerialVL, a large-scale dataset, which is collected using UAV flying at different altitudes, along various routes, and during diverse time periods. AerialVL consists of 11 image sequences covering approximately 70 km of trajectory and includes a reference satellite image database corresponding to the flight area. Leveraging AerialVL, we perform thorough evaluations on various mainstream solutions designed for aerial-based visual localization for the first time. This evaluation encompasses visual place recognition, visual alignment localization and visual odometry, serving as comparison baselines. Furthermore, we present a general aerial-based visual localization framework, which unifies various methods and integrates them into a modular architecture. We note that across all flight trajectories, the proposed framework achieves higher localization accuracy and robustness against the existing methods.
23 September 2025 in IEEE Transactions on Robotics
This article introduces GeoVINS, a vision-based navigation framework designed for large-scale global state estimation. By utilizing geographic information from satellite orthoimagery, GeoVINS tackles scale ambiguity and accumulative drift problems inherent in standard visual-inertial simultaneous localization and mapping (VI-SLAM) systems, and provides accurate, robust, and real-time global localization. In particular, to address the challenge of memory explosion issue for large-scale localization, we propose a novel aerial ‘classify-then-retrieve’ aerial VPR approach, where geographic locations can be efficiently identified and memory usage can be reduced by 3 to 4 orders of magnitude compared to classical retrieval-based approaches. In addition, a hierarchical geographic data association scheme, enhanced by state-of-the-art deep learning-based feature matching, guarantees high efficiency and robustness against variations in appearance and viewpoint. Relying on the obtained three-dimensional (3D) geographic information, GeoVINS achieves efficient global state initialization and precise motion tracking. To address GPU limitations in embedded devices, a collaborative CPU-GPU utilization approach is proposed, seamlessly integrating asynchronous global information to eliminate accumulative errors. Relying solely on satellite imagery and without requiring any other prior information, GeoVINS achieves rapid place recognition in unseen environments of city-scale (e.g., 2500${\mathrm{km}}^{2}$) on an embedded computing device with an inference time of 43 ms, and performs state estimation at a frequency of 25 Hz. The system enables autonomous UAV navigation as an alternative to GNSS.