In this study, we present a novel, robust, and accurate method to register multiple images captured by an unmanned aerial vehicle (UAV) with modified camera matrixes, and then stitch UAV images to a seamless one. Firstly, the camera parameters, camera poses and sparse 3D points are solved using feature correspondence and SFM. Secondly, all images are registered using sparse 3D point cloud and modified camera matrixes. Thirdly, the stitching plane is determined by performing a view selection. At last, all images are stitched using multi-band blending based on graph-cut. Experiments and comparisons demonstrate the accuracy and feasibility of our approach.

Recovering 3D structure of the scene is widely studied in computer vision, which is called structure from motion. Most of the structure from motion approaches using features points to reconstruct 3D point cloud maps to represent scenes. However, there are always many lines in manmade scenes. Line is a higher-level representation of scene structure than point. We contribute a SLAM (simultaneous location and mapping) based method to reconstruct the 3D lines of urban scenes to emphasize the structures. In our approach, keyframe based SLAM system provides keyframes and camera poses, line segments are extracted and tracked in keyframes, the spatial line is determined in two steps: direction then position. The direction is acquired from normal vectors of multiple line projection planes and depth is searched by minimizing the total re-projection error of a spatial line in multiple keyframes. Degenerate and incorrectly tracked cases are detected by rank and principal components analysis. We present accurate 3D line model for urban scene with UAV inspection. The generated 3D lines give a better structural sense than sparse point cloud.

Conventional initialization algorithms for feature-based monocular visual Simultaneous Localization and Mapping(V-SLAM)/Visual Odometry(VO) systems used point-based matrix decomposition. However, line segments are common in urban scenes and provide valuable information about scene structure. In this work, we propose incorporating line segments for initializing, which allows to obtain accurate initial camera poses and generate reliable 3D line segment landmarks as well as point landmarks. The proposed algorithm, denoted as PLSfSM (Point and Line Structure from Small Motion), employs both point and Plücker line reprojection constraints to recover structure from small motion. In particular, we propose a closed-form solution to estimate rotation and optimize rotation with line segment correspondences, which provides good initial guesses for rotation and line directions. Mathematically proven criteria are introduced to detect degeneracy of line reconstruction. The algorithm is evaluated on synthetic indoor dataset and simulation data. Qualitative experiments on both indoor and outdoor scenes are conducted to verify the robustness. Experimental results validate that the proposed algorithm which combines both points and line segments for initialization outperforms the state-of-the-art methods which only use points for initialization

The loop-closure detection is an essential part in visual SLAM. This paper proposes an effective approach to generate high-level semantic image features for improving loop-closure detection and place recognition, based on a hybrid Convolutional Neural Network (CNN) modified by us to especially cope with real-time feature extraction. This hybrid deep learning architecture is adapted from ResNet-18 and enhanced by split-transform-merge idea as well as squeeze-and-excitation structure, such that the network's ability of image representation can be compensated without compromising on efficiency. Then, we design a simple yet well-fitted compression strategy for reducing the dimension of deep convolutional features. The strategy significantly accelerates the speed of image-pairs matching with little loss or even improvements in Precision/Recall performance. Our approach is validated on public benchmark datasets and compared to state-of-the-art methods. Experiments demonstrate improved Precision/Recall outperforming state of the art. Additionally, that precision declines very slow with the increasing of recall indicates potential of further progress.

Line segments are common in urban scenes and contain rich structural information. However, different from point-based reconstruction, reconstructed 3D lines may have large displacement from the ground truth in spite of a very small sum of reprojection error. In this work, we present a method to analyze the uncertainty of line reconstruction and provide a quantitative evaluation of accuracy. A new minimal four-vector line representation based on Plücker line is introduced, which is tailed for uncertainty analysis of line reconstruction. Each component of the compact representation has a certain physical meaning about the 3D line’s orientation or position. The reliability of the reconstructed lines can be measured by the confidence interval of each component in the proposed representation. Based on the uncertainty analysis of two-view line triangulation, the uncertainty of multi-view line reconstruction is also derived. Combining the proposed uncertainty analysis with reprojection error, a more reliable 3D line map can be obtained. Experiments on simulation data, synthetic and real-world image sequences validate the feasibility of the proposed method.