Visual inertial odometry (VIO) is a technique to estimate the change of a mobile platform in position and orientation overtime using the measurements from on-board cameras and IMU sensor. Recently, VIO attracts significant attentions from large number of researchers and is gaining the popularity in various potential applications due to the miniaturisation in size and low cost in price of two sensing modularities. However, it is very challenging in both of technical development and engineering implementation when accuracy, real-time performance, robustness and operation scale are taken into consideration. This survey is to report the state of the art VIO techniques from the perspectives of filtering and optimisation-based approaches, which are two dominated approaches adopted in the research area. To do so, various representations of 3D rigid motion body are illustrated. Then filtering-based approaches are reviewed, and followed by optimisation-based approaches. The links between these two approaches will be clarified via a framework of the Bayesian Maximum A Posterior. Other features, such as observability and self calibration, will be discussed. Graphical Abstract

This paper presents an indoor relocalization system using a dual-stream convolutional neural network (CNN) with both color images and depth images as the network inputs. Aiming at the pose regression problem, a deep neural network architecture for RGB-D images is introduced, a training method by stages for the dual-stream CNN is presented, different depth image encoding methods are discussed, and a novel encoding method is proposed. By introducing the range information into the network through a dual-stream architecture, we not only improved the relocalization accuracy by about 20% compared with the state-of-the-art deep learning method for pose regression, but also greatly enhanced the system robustness in challenging scenes such as large-scale, dynamic, fast movement, and night-time environments. To the best of our knowledge, this is the first work to solve the indoor relocalization problems based on deep CNNs with RGB-D camera. The method is first evaluated on the Microsoft 7-Scenes data set to show its advantage in accuracy compared with other CNNs. Large-scale indoor relocalization is further presented using our method. The experimental results show that 0.3 m in position and 4° in orientation accuracy could be obtained. Finally, this method is evaluated on challenging indoor data sets collected from motion capture system. The results show that the relocalization performance is hardly affected by dynamic objects, motion blur, or night-time environments.

The recent advances in unmanned aerial vehicles (UAVs) enormously improve their utility and expand their application scope. The UAV and swarm implementation further prevail in Smart City practices with the aid of edge computing and urban Internet of Things. The lead–follow formation in UAV swarm is an important organization means and has been adopted in diverse exercises, for its efficiency and ease of control. However, the reliability of centralization makes the entire swarm system in risk of collapse and instability, if a fatal fault incident happens in the leader. The motivation is to build a mechanism helping the distributed swarm recover from possible failures. Existing ways include assigning definite backups, temporary clustering and traversing to select a new leader are traditional ways that lack flexibility and adaptability. In this article, we propose a voting-based leader election scheme inspired by the Raft method in distributed computation consensus to solve the problem. We further discuss the impact of communication conditions imposed on the decentralized voting process by implementing a network resource pool. To dynamically evaluate UAV individuals, we outline measurement design principles and provide a realizable calculation example. Lastly but not least, empirical simulation results manifest better performance than the Raft-based method. Our voting-based approach exhibits advantages and is a promising way for quick regrouping and fault recovery in lead–follow swarms.

In multi-UAVs system, high-quality communication system is the key to ensure cooperation. Most of the existing methods focus on the trade-off between connectivity and the number of network edges. However, for a multi-UAVs communication network with high quality of service (QoS), more metrics of communication network need to be considered, such as the hops and the maximum number of links. In this paper, we propose a multi-objective optimization method for dynamic communication network. Firstly, in order to accurately estimate the moved positions with random noise, Long and Short Term Memory network (LSTM) is used to predict succeeding positions of all UAVs. Secondly, Deep Q-network (DQN) with random action space is designed to adaptively adjust the communication network. In particular, it can be applied to large-scale communication networks. Furthermore, we propose a transfer learning method based on the trained LSTM and DQN. In this transfer learning, these two models which are well-trained in smaller communication network and combined with principal component analysis (PCA) can be directly used for communication network optimization of larger networks. Experimental results show that our method is not only superior to other methods but also faster in convergence.

In this letter, we propose a robust cross-drone multitarget association algorithm that aims to associate the identities of targets detected by different drones with overlapping FOVs. Unlike other methods using image features (target appearance features and background features), our approach makes full use of the three-dimensional spatial consistency (TSC) between targets. It has wider applicability in deployed overlapping crossdrone systems with unreliable image features. Specifically, we first explore target correlation from the perspective of three dimensional space, and propose an association method that integrates position and altitude consistency constraints, without relying on image features. After that, we mine topological mapping relationships (TMR) between cameras from the preliminary results and propose a TMR-based re-association approach using cost evaluation to optimize the preliminary results, which effectively improve association accuracy. Extensive experimental evaluations on Airsim-based dataset verify the effectiveness and robustness of our proposed method. The code is available at https://github.com/pantingwei/TSCTMR .

This paper develops a framework for geolocating a ground moving target with images taken from an unmanned aerial vehicle (UAV). Unlike the usual moving target geolocation approaches that rely heavily on a laser rangefinder, multiple UAVs, prior information of the target or motion assumptions, the proposed framework performs the geolocation of a moving target with monocular vision and does not have any of the above restrictions. The proposed framework transforms the problem of moving target geolocation to the problem of stationary target geolocation by matching corresponding points. In the process of corresponding point matching, we first propose a Siamese-network-based model as the base model to match corresponding points between the current frame and the past frame. Besides the introduction of a base model, we further designed an enhanced model with two outputs, where a row-ness loss and a column-ness loss are defined for achieving a better performance. For the precision of corresponding point matching, we propose a compensation value, which is calculated from the outputs of the enhanced model and improves the accuracy of corresponding point matching. To facilitate the research on corresponding point matching, we constructed a dataset containing various aerial images with corresponding point annotations. The proposed method is shown to be valid and practical via the experiments in simulated and real environments.