Simultaneous Localization and Mapping (SLAM) is a crucial technology for intelligent unnamed systems to estimate their motion and reconstruct unknown environments. However, the SLAM systems with merely one sensor have poor robustness and stability due to the defects in the sensor itself. Recent studies have demonstrated that SLAM systems with multiple sensors, mainly consisting of LiDAR, camera, and IMU, achieve better performance due to the mutual compensation of different sensors. This paper investigates recent progress on multi-sensor fusion SLAM. The review includes a systematic analysis of the advantages and disadvantages of different sensors and the imperative of multi-sensor solutions. It categorizes multi-sensor fusion SLAM systems into four main types by the fused sensors: LiDAR-IMU SLAM, Visual-IMU SLAM, LiDAR-Visual SLAM, and LiDAR-IMU-Visual SLAM, with detailed analysis and discussions of their pipelines and principles. Meanwhile, the paper surveys commonly used datasets and introduces evaluation metrics. Finally, it concludes with a summary of the existing challenges and future opportunities for multi-sensor fusion SLAM.
19 February 2018 in IEEE Transactions on Industrial Electronics
This paper develops a method of calculating the three-dimensional (3-D) coordinates of a target with image information taken from an unmanned aerial vehicle (UAV). Unlike the usual approaches to target geolocation that rely heavily on georeferenced terrain database or accurate attitude sensors, the contribution of the proposed method is to offset the constraints and also perform 3-D geolocation of the target based on the relative altitude between the aircraft and the target calculated using the stereo vision technique. Considering the poor performance of the yaw-angle measurement provided by the low-quality attitude sensors in the UAV, a novel vision-based 3-D geolocation method is designed. The proposed method is proven to be valid and practical via simulation results and actual flight experiments.
7 April 2020 in IEEE Transactions on Image Processing
Binocular vision is a passive method of simulating the human visual principle to perceive the distance to a target. Traditional binocular vision applied to target localization is usually suitable for short-range area and indoor environment. This paper presents a novel vision-based geolocation method for long-range targets in outdoor environment, using handheld electronic devices such as smart phones and tablets. This method solves the problems in long-range localization and determining geographic coordinates of the targets in outdoor environment. It is noted that these sensors necessary for binocular vision geolocation such as the camera, GPS, and inertial measurement unit (IMU), are intergrated in these handheld electronic devices. This method, employing binocular localization model and coordinate transformations, is provided for these handheld electronic devices to obtain the GPS coordinates of the targets. Finally, two types of handheld electronic devices are used to conduct the experiments for targets in long range up to 500m. The experimental results show that this method yields the target geolocation accuracy along horizontal direction with nearly 20m, achieving comparable or even better performance than monocular vision methods.
12 April 2022 in IEEE Transactions on Industrial Electronics
To solve the problem of collaborative air-ground target search and localization in large outdoor environments, we adopt a distributed layered architecture and proposed a corresponding distributed estimation algorithm, which can produce accurate estimates of interested target position. The air-ground system consists of a single unmanned aerial vehicle and several ground platforms. Firstly, for the distributed layered architecture, this paper develops unbalanced simple convex combination algorithm using one-shot measurements from the single unmanned aerial vehicle and all ground platforms to generate global estimates of the stationary target state. The novelty of the proposed algorithm is that it considers the differences in sensor capabilities between the aerial and ground platforms. Then, performance analysis is described to show that the proposed algorithm is obviously superior to the traditional existing convex combination algorithms. Finally, the efficacy of this algorithm is further demonstrated by the simulation experiments and actual geolocation tests.
13 December 2018 in IEEE Transactions on Aerospace and Electronic Systems
This paper describes a novel method to improve the target location accuracy through imaging it from an aircraft. This method focuses on improving estimation accuracy of heading angle bias to then improve geolocation performance. A Particle Swarm Optimization (PSO) algorithm is employed to derive an expression of optimal trajectory, which can be a guide for trajectory planning. Thanks to the maneuverability of quad-rotor unmanned aerial vehicles (UAV), the aircraft is commanded to follow path generated by trajectory planning to acquire multiple bearing measurements of the ground object. The main result is that the aircraft's heading angle bias can be more accurately estimated using trajectory planning. Hence, the target is more accurately geolocated. The efficacy of this technique is verified and demonstrated by simulation results and flight test.
We present an air-to-air multi-sensor and multi-view fixed-wing UAV dataset, MMFW-UAV, in this work. MMFW-UAV contains a total of 147,417 fixed-wing UAVs images captured by multiple types of sensors (zoom, wide-angle, and thermal imaging sensors), displaying the flight status of fixed-wing UAVs of different sizes, appearances, structures, and stabilized flight velocities from multiple aerial perspectives (top-down, horizontal, and bottom-up views), aiming to cover the full-range of perspectives with multi-modal image data. Quality control processes of semi-automatic annotation, manual check, and secondary refinement are performed on each image. To the best of our knowledge, MMFW-UAV is the first one-to-one multi-modal image dataset for fixed-wing UAVs with high-quality annotations. Several mainstream deep learning-based object detection architectures are evaluated on MMFW-UAV and the experimental results demonstrate that MMFW-UAV can be utilized for fixed-wing UAV identification, detection, and monitoring. We believe that MMFW-UAV will contribute to various fixed-wing UAVs-based research and applications.