Visual simultaneous localization and mapping (v-SLAM) and navigation of unmanned aerial vehicles (UAVs) are receiving increasing attention in both research and education. However, extensive physical testing can be expensive and time-consuming due to safety precautions, battery constraints, and the complexity of hardware setups. For the efficient development of navigation algorithms and autonomous systems, as well as for education purposes, the ROS-Gazebo-PX4 simulator was customized in-depth, integrated into our previous released research works, and provided as an end-to-end simulation (E2ES) solution for UAV, v-SLAM, and navigation applications. Unlike most other similar works, which can only stimulate certain parts of the navigation algorithms, the E2ES platform simulates all of the localization, mapping, and path-planning kits in one simulator. The navigation stack performs well in the E2ES test bench with the absolute pose errors of 0.3 m (translation) and 0.9 degree (rotation), respectively, for an 83 m length trajectory. Moreover, the E2ES provides an out-of-box, click-and-fly autonomy in UAV navigation. The project source code is opened for the benefit of the research community.

In this article, we present a stereo visual inertial pose estimation method based on feedforward and feedbacks. Compared to the widely used filter-based or optimization-based approaches, the proposed method only stores the most recent pose and measurements and thus can achieve fast processing. A gradient decreased feedback, a roll-pitch feedforward, and a bias estimation feedback, are introduced to fuse the vision and the inertial measurements. This system, which is called feedforward and feedback based visual inertial system (FVIS), is evaluated on the popular European robotics challenge micro aerial vehicle (EuRoC MAV) dataset. FVIS achieves high accuracy and robustness with respect to existing visual inertial simultaneous localization and mapping (SLAM) approaches. FVIS has also been implemented and tested on a unmanned aerial vehicle (UAV) platform. The source code developed during this study is available publicly.

The inspection of electrical and mechanical (E&M) devices using unmanned aerial vehicles (UAVs) has become an increasingly popular choice in the last decade due to their flexibility and mobility. UAVs have the potential to reduce human involvement in visual inspection tasks, which could increase efficiency and reduce risks. This paper presents a UAV system for autonomously performing E&M device inspection. The proposed system relies on learning-based detection for perception, multi-sensor fusion for localization, and path planning for fully autonomous inspection. The perception method utilizes semantic and spatial information generated by a 2-D object detector. The information is then fused with depth measurements for object state estimation. No prior knowledge about the location and category of the target device is needed. The system design is validated by flight experiments using a quadrotor platform. The result shows that the proposed UAV system enables the inspection mission autonomously and ensures a stable and collision-free flight.

This work presents the implementation of system identification and model predictive control for a tail-sitter unmanned aerial vehicle (UAV) in cruise flight. The mathematical model of longitudinal and lateral directions of the UAV has been derived in the state-space form for grey-box modeling. The least-square regression method is augmented with regulation and solved by applying the trust-region algorithm. Outdoor flight tests were conducted to acquire the data for system identification assisted by a signal generator module. The UAV dynamic was sufficiently excited in both longitudinal and lateral directions during the flight test. The flight data were applied to the grey box system identification, and the parameters were validated by fitting the reconstructed model to a set of flight data with a different excitation waveform. The flight controller with model predictive control was formed using the identified models for flight simulation. The results demonstrate that the system identification results are able to provide reference models for the model-based controller development of the novel-design tail-sitter UAV.

Mapping, as the back-end of perception and the front-end of path planning in the modern UAV navigation system, draws our interest. Considering the requirements of UAV navigation and the features of the current embedded computation platforms, we designed and implemented a novel multilayer mapping framework. In this framework, we divided the map into three layers: awareness, local, and global. The awareness map is constructed in cylindrical coordinate, enabling fast raycasting. The local map is a probability-based volumetric map. The global map adopts dynamic memory management, allocating memory for the active mapping area, and recycling the memory from the inactive mapping area. We implemented this mapping framework in three parallel threads: awareness thread, local-global thread, and visualization thread. Finally, we evaluated the mapping kit in both the simulation environment and the real-world scenario with the vision-based sensors. The framework supports different kinds of map outputs for the global or local path planners. The implementation is open-source for the research community.

Visual inertial odometry (VIO) is the front-end of visual simultaneous localization and mapping (vSLAM) methods and has been actively studied in recent years. In this context, a time-of-flight (ToF) camera, with its high accuracy of depth measurement and strong resilience to ambient light of variable intensity, draws our interest. Thus, in this paper, we present a realtime visual inertial system based on a low cost ToF camera. The iterative closest point (ICP) methodology is adopted, incorporating salient point-selection criteria and a robustness-weighting function. In addition, an error-state Kalman filter is used and fused with inertial measurement unit (IMU) data. To test its capability, the ToF–VIO system is mounted on an unmanned aerial vehicle (UAV) platform and operated in a variable light environment. The estimated flight trajectory is compared with the ground truth data captured by a motion capture system. Real flight experiments are also conducted in a dark indoor environment, demonstrating good agreement with estimated performance. The current system is thus shown to be accurate and efficient for use in UAV applications in dark and Global Navigation Satellite System (GNSS)-denied environments.