24 June 2024 in IEEE Transactions on Instrumentation and Measurement
The autonomous flight of Unmanned Aerial Vehicle (UAV) relies on precise and robust geo-localization system. Visual geo-localization system registers the aerial captured image to a geo-referenced satellite map, which enables UAVs to determine their global position without Global Navigation Satellite System (GNSS). However, it is challenging to achieve precise and robust geo-localization in large-scale environment due to notable texture difference between the satellite map and the UAV-captured image. In this paper, we design a robust visual geo-localization pipeline which integrates a proposed deep learning-based imagery feature. This pipeline starts with the image retrieval based on the deep feature encoding, to initialize the localization process over large-scale maps without any location prior. With the reuse of the same deep imagery feature, an image registration process enables the real-time sequential localization. Besides, the proposed system has the re-localization ability to eliminate the localization drift caused by the possible registration failure, especially during the long-time flight. Evaluations on datasets and real-world experiments demonstrate that the proposed system is more robust and accurate than other state-of-the-art methods. To encourage the further progress on visual geo-localization problem, our code and materials are publicly available at https://github.com/hmf21/ UAVLocalization.
2 October 2023 in IEEE Transactions on Intelligent Transportation Systems
This paper presents an efficient and accurate range-aided visual-inertial odometry (RVIO) system for the global positioning system denied environment. In particular, the ultra-wideband (UWB) measurements are integrated to reduce the long-term drift of the visual-inertial odometry (VIO) system. Our approach starts with a filter-based scheme to localize the unknown UWB anchor in the local world frame. In particular, a novel surface-based particle filter is proposed to localize the UWB anchors efficiently. When the initialization is complete, the UWB location information is utilized to support the subsequent long-term robot positioning. An observability-constrained optimization approach is developed to combine the visual, inertial, and UWB range measurements. Such a framework takes advantage of both VIO and UWB measurements and is feasible even when the number of observed UWB anchors is below four. Experiments on both simulated and real-world scenes demonstrate the validity and superiority of the proposed system.
25 April 2023 in IEEE Robotics and Automation Letters
Vision-based localization is an essential problem for autonomous systems while the performance of vision-based odometry degrades in the challenging scenario. This letter presents S-VIO, an RGB-D visual inertial odometry (VIO) which fully uses multi-sensor measurements (i.e. depth, RGB and IMU), heterogeneous landmarks (i.e. points, lines and planes) and structural regularity of the environment to obtain a robust and accurate localization result. In order to detect the underlying structural regularity of the environment, a two-step Atlanta world inference method is proposed. Leveraging the gravity direction estimated by the VIO system, the proposed algorithm first generates horizontal Atlanta axis hypotheses from a set of recently optimized plane landmarks. Then the following plane landmarks and line clusters are used to filter out the occasionally observed axes based on the persistence of the observation. The remaining axes will survive and be saved in the Atlanta map for future re-observations. Particularly, an efficient mined-and-stabbed (MnS) method is applied to classify the structural line and extract the vanishing point from each line cluster. In addition, a closed-form initialization method for the structural line feature is proposed, which leverages the known direction to obtain a better initial estimation. Integrated with the above novelties, S-VIO is tested on two public real-world RGB-D inertial datasets. Experiments demonstrate that S-VIO has better accuracy and robustness compared to state-of-the-art VIO and RGB-D VIO algorithms.
20 September 2024 in IEEE Robotics and Automation Letters
Visual inertial odometry (VIO) is a popular solution for localization in the GPS-denied environment. However, current VIO algorithms' performance degrades in the visual degraded environment due to limited visual features. This paper presents a robust VIO algorithm with learned velocity constraints. In particular, an encoder-decoder neural network augmented with the attention mechanism is employed to estimate the instant velocity and its uncertainty based on the IMU measurement. Then the learned velocities serve as the pseudo-measurement and fused with the inertial and visual measurement within a standard optimization framework to obtain an accurate and robust localization result. The key idea of this work is that the velocity constraint learned from the inertial measurement can effectively prevent the divergence of the VIO in the visual degraded environment without introducing any new sensors and better model the short-term characteristics of the motion compared to the previous neural VIO algorithms based on the displacement vector. Experiments on the public dataset demonstrate that the proposed VIO algorithm operates robustly under the low-visibility condition and obtains higher localization accuracy compared with other state-of-the-art algorithms. We will share code and models.
23 May 2025 in IEEE Robotics and Automation Letters
Image-goal navigation is a critical task in autonomous visual navigation, requiring the robot to navigate to a target localization specified by an image. Previous works using data-driven methods achieve great success while they mostly leverage simple network architecture and train it from scratch, which limits the navigation performance in challenging situations, such as multiple turns or varying lighting conditions. In this letter, we thoroughly analyze the essential features for visual navigation and design an effective network to achieve optimal navigation performance. In particular, we leverage a pretrained foundation model for feature extraction, introduce cross attention for goal encoding and propose a token attention mechanism to dynamically assign weights to different tokens. The proposed model achieves excellent navigation performance in unseen environments. Experiments in real world demonstrate that our method achieves a success rate of 87%, 40% improvement over the state-of-the-art methods.
6 September 2024 in IEEE Transactions on Intelligent Vehicles
In this paper, we present an inertial-wheel odometry leveraging the learning-based yaw velocity estimations for autonomous vehicles. A novel attention-based temporal convolutional network, namely YNet, is employed to estimate yaw velocities by integrating multimodal data from the Inertial Measurement Unit (IMU) and wheel encoders. These learning-based estimations are then integrated into a state-of-the-art invariant extended Kalman filter along with the velocity measurements from wheel encoders. Impressive long-term localization performance is achieved by relying solely on motion measurement sensors, namely IMU and wheel encoders. This characteristic grants it remarkable robustness in diverse and complex environments, surpassing the limitations of vision or lidar based odometry methods. The evaluations on the Kaist datasets and real-world experiments demonstrate that the proposed approach effectively reduces the drift of long-term inertial localization, yielding superior results compared to other state-of-the-art methods