The autonomous flight of Unmanned Aerial Vehicle (UAV) relies on precise and robust geo-localization system. Visual geo-localization system registers the aerial captured image to a geo-referenced satellite map, which enables UAVs to determine their global position without Global Navigation Satellite System (GNSS). However, it is challenging to achieve precise and robust geo-localization in large-scale environment due to notable texture difference between the satellite map and the UAV-captured image. In this paper, we design a robust visual geo-localization pipeline which integrates a proposed deep learning-based imagery feature. This pipeline starts with the image retrieval based on the deep feature encoding, to initialize the localization process over large-scale maps without any location prior. With the reuse of the same deep imagery feature, an image registration process enables the real-time sequential localization. Besides, the proposed system has the re-localization ability to eliminate the localization drift caused by the possible registration failure, especially during the long-time flight. Evaluations on datasets and real-world experiments demonstrate that the proposed system is more robust and accurate than other state-of-the-art methods. To encourage the further progress on visual geo-localization problem, our code and materials are publicly available at https://github.com/hmf21/ UAVLocalization.

Visual localization plays an essential role in the autonomous flight of Unmanned Aerial Vehicles (UAVs) especially for the Global Navigation Satellite System (GNSS) denied environments. Existing aerial-based visual localization methods mainly focus on eliminating image variance between database map and captured frames. However, these is a lack of public dataset and baseline for method comparisons, which impedes the development of aerial-based visual localization. To address this issue, we construct AerialVL, a large-scale dataset, which is collected using UAV flying at different altitudes, along various routes, and during diverse time periods. AerialVL consists of 11 image sequences covering approximately 70 km of trajectory and includes a reference satellite image database corresponding to the flight area. Leveraging AerialVL, we perform thorough evaluations on various mainstream solutions designed for aerial-based visual localization for the first time. This evaluation encompasses visual place recognition, visual alignment localization and visual odometry, serving as comparison baselines. Furthermore, we present a general aerial-based visual localization framework, which unifies various methods and integrates them into a modular architecture. We note that across all flight trajectories, the proposed framework achieves higher localization accuracy and robustness against the existing methods.

In Global Navigation Satellite System (GNSS)-denied environments, image registration has emerged as a prominent approach to utilize visual information for estimating the position of Unmanned Aerial Vehicles (UAVs). However, traditional image-registration-based localization methods encounter limitations, such as strong dependence on the prior initial position information. In this paper, we propose a systematic method for UAV geo-localization. In particular, an efficient range–visual–inertial odometry (RVIO) is proposed to provide local tracking, which utilizes measurements from a 1D Laser Range Finder (LRF) to suppress scale drift in the odometry. To overcome the differences in seasons, lighting conditions, and other factors between satellite and UAV images, we propose an image-registration-based geo-localization method in a coarse-to-fine manner that utilizes the powerful representation ability of Convolutional Neural Networks (CNNs). Furthermore, to ensure the accuracy of global optimization, we propose an adaptive weight assignment method based on the evaluation of the quality of image-registration-based localization. The proposed method is extensively evaluated in both synthetic and real-world environments. The results demonstrate that the proposed method achieves global drift-free estimation, enabling UAVs to accurately localize themselves in GNSS-denied environments.

This article introduces GeoVINS, a vision-based navigation framework designed for large-scale global state estimation. By utilizing geographic information from satellite orthoimagery, GeoVINS tackles scale ambiguity and accumulative drift problems inherent in standard visual-inertial simultaneous localization and mapping (VI-SLAM) systems, and provides accurate, robust, and real-time global localization. In particular, to address the challenge of memory explosion issue for large-scale localization, we propose a novel aerial ‘classify-then-retrieve’ aerial VPR approach, where geographic locations can be efficiently identified and memory usage can be reduced by 3 to 4 orders of magnitude compared to classical retrieval-based approaches. In addition, a hierarchical geographic data association scheme, enhanced by state-of-the-art deep learning-based feature matching, guarantees high efficiency and robustness against variations in appearance and viewpoint. Relying on the obtained three-dimensional (3D) geographic information, GeoVINS achieves efficient global state initialization and precise motion tracking. To address GPU limitations in embedded devices, a collaborative CPU-GPU utilization approach is proposed, seamlessly integrating asynchronous global information to eliminate accumulative errors. Relying solely on satellite imagery and without requiring any other prior information, GeoVINS achieves rapid place recognition in unseen environments of city-scale (e.g., 2500${\mathrm{km}}^{2}$) on an embedded computing device with an inference time of 43 ms, and performs state estimation at a frequency of 25 Hz. The system enables autonomous UAV navigation as an alternative to GNSS.

Aerial Visual Place Recognition (VPR) is critical for Autonomous Aerial Vehicles (UAVs) localization, especially in environments with unstable or unavailable GPS signals. While neural network-based VPR methods have become mainstream, they face significant challenges on UAV platforms. Traditional CNN-based VPR models are highly sensitive to image rotation, degrading their performance in aerial-domain environments. Meanwhile, Transformer-based models have high computational complexity, making them less suitable for resource-constrained UAVs. In this letter, we propose a lightweight, rotation-invariant aerial VPR method. Our approach combines a rotation-equivariant backbone network with a rotation-invariant aggregation layer to ensure descriptor consistency across different orientations. Additionally, we propose an unsupervised training strategy that constructs higher-dimensional descriptors to optimize the model, while maintaining the lower descriptor dimensionality during application. Experimental results show that our method outperforms state-of-the-art methods across multiple aerial VPR datasets.

Visual Place Recognition (VPR) is a critical technology for achieving robust long-term visual geo-localization. During the past few years, VPR research mainly focused on ground-based platforms in the street-level captured scenes with deep learning methods (e.g. NetVLAD, GeM), but little attention was paid to the VPR task on aerial vehicles. The algorithms and models designed for ground-based platforms are always directly applied to the aerial VPR problem. However, the viewpoint variance on Unmanned Aerial Vehicles (UAV) is much larger than the ground-based platforms. Due to the sparse distribution of aerial image features, when the viewpoint of the camera changes, the features of the query image are largely inconsistent with the descriptors in the database, which results in the failures of image retrieval and visual geo-localization. In this letter, we propose an aerial VPR enhancement module called GeoCluster , which presents a feature aggregation method using spatial clustering information to improve the robustness and consistency of the global descriptors for UAV-captured frames. Moreover, it can be applied to any NetVLAD-based VPR method and boost the pre-trained model without any further training process. By integrating GeoCluster into an existing state-of-the-art localization method, we can achieve about 10% improvement for aerial image retrieval tasks and have more accurate and robust geo-localization results.