Inspired by simultaneous localization and mapping (SLAM) style workflow, this article presented an online sequential structure from motion (SfM) solution for high-frequency video and large baseline high-resolution aerial images with high efficiency and novel precision. First, as traditional SLAM systems are not good in processing low overlap images, based on our novel hierarchical feature matching paradigm with multihomography and BoW, we proposed a robust tracking method where the relative pose and its scale are estimated separately followed by a joint optimization by considering both perspective-n-point (PnP) and epipolar constraints. Second, to further optimize the camera poses for the sparse map and dense pointcloud reconstruction, we provided a graph-based optimization with reprojection and GPS constraints, which make the camera trajectory and map georeferenced. We also incrementally generated the dense point cloud in real time from keyframes after local mapping optimization. Finally, we use a publicly available aerial image dataset with sequences of different environments, to evaluate the effectiveness of the proposed method, meanwhile, the robust performance of our solution is demonstrated with applications of high-quality aerial images mosaic and digital surface model (DSM) reconstruction in real time. Compared with the state-of-the-art SLAM and traditional SfM methods, the presented system can output large-scale high-quality ortho-mosaic and DSM in real time with the low computational cost.

Individual tree counting (ITC) is a popular topic in the remote sensing application field. The number and planting density of trees are significant for estimating the yield and for futher planing, etc. Although existing studies have already achieved great performance on tree detection with satellite imagery, the quality is often negatively affected by clouds and heavy fog, which limits the application of high-frequency inventory. Nowadays, with ultra high spatial resolution and convenient usage, Unmanned Aerial Vehicles (UAVs) have become promising tools for obtaining statistics from plantations. However, for large scale areas, a UAV cannot capture the whole region of interest in one photo session. In this paper, a real-time orthophoto mosaicing-based tree counting framework is proposed to detect trees using sequential aerial images, which is very effective for fast detection of large areas. Firstly, to guarantee the speed and accuracy, a multi-planar assumption constrained graph optimization algorithm is proposed to estimate the camera pose and generate orthophoto mosaicing simultaneously. Secondly, to avoid time-consuming box or mask annotations, a point supervised method is designed for tree counting task, which greatly speeds up the entire workflow. We demonstrate the effectiveness of our method by performing extensive experiments on oil-palm and acacia trees. To avoid the delay between data acquisition and processing, the proposed framework algorithm is embedded into the UAV for completing tree counting tasks, which also reduces the quantity of data transmission from the UAV system to the ground station. We evaluate the proposed pipeline using sequential UAV images captured in Indonesia. The proposed pipeline achieves an F1-score of 98.2% for acacia tree detection and 96.3% for oil-palm tree detection with online orthophoto mosaicing generation.

Orthophoto generation is a popular topic in aerial photogrammetry and 3D reconstruction. It is generally computationally expensive with large memory consumption. Inspired by the simultaneous localization and mapping (SLAM) workflow, this paper presents an online sequential orthophoto mosaicing solution for large baseline high-resolution aerial images with high efficiency and novel precision. An appearance and spatial correlation-constrained fast low-overlap neighbor candidate query and matching strategy is used for efficient and robust global matching. Instead of estimating 3D positions of sparse mappoints, which is outlier sensitive, we propose to describe the ground reconstruction with multiple stitching planes, where parameters are reduced for fast nonconvex graph optimization. GPS information is also fused along with six degrees of freedom (6-DOF) pose estimation, which not only provides georeferenced coordinates, but also converges property and robustness. An incremental orthophoto is generated by fusing the latest images with adaptive weighted multiband algorithm, and all results are tiled with level of detail (LoD) support for efficient rendering and further disk cache for reducing memory usages. Public datasets are evaluated by comparing state-of-the-art software. Results show that our system outputs orthophoto with novel efficiency, quality, and robustness in real-time. An android commercial application is developed for online stitching with DJIdrones, considering the excellent performance of our algorithm.

A georeferenced ortho-photo built from aerial images is the basic resource for various of remote sensing applications. As traditional low flight altitude aerial photographic survey is hard to handle large areas, higher altitude brings better survey efficiency and a telephoto camera is required to maintain the resolution. Since existed structure from motion (SfM) methods are not suitable to process low field-of-view images with high altitude, this paper presents a novel georeferenced ortho-photo stitching solution for telephoto images which contain both translation and rotation movements. A mission planning strategy is designed to guarantee the consistency quality according to the flight plan and camera intrinsic parameters. The potential matching topology is obtained by projecting images to the prior ground plane and overlap scores are estimated for neighbors query, followed with a feature based keypoint matching. The ground plane and camera poses are optimized by joint considering feature matches, GPS and gimbal rotation informations, while the final orthophoto is fused with a fast exposure leveling and weighted multi-band blending algorithm. The experiments show that our system is able to output novel quality orthophoto with high robustness, and we shared a trial version of our algorithm on website: \url{https://github.com/zdzhaoyong/highstitch}.

An efficient and precise three-dimensional shape registration method using the continuous distance field is described, which reformulates the point set registration as the nonlinear least squares optimization problem. The continuous distance field is proposed by the approximate distance function derived from the implicit formulation of the target model. The registration error with respect to the relative pose is constructed by the sum of the squared distance between the source set to the target model. In this paper, the twist coordinates are adopted to represent the rigid transformation of the 3D object during the registration process. With the property of Lie group and Lie algebra, the optimal registration is formulated concisely as an unconstrained optimization problem. In this way, the best match can be achieved by directly optimizing the six pose parameters of the object. The detailed formulation of the Gauss-Newton method for solving the nonlinear least squares problem is explicitly derived. Experimental results show the proposed approach is capable of aligning the 3D free-form object with faster convergence speed, especially in the presence with a large amount of point cloud, which demonstrates the potential applications of the method such as pose estimation and object tracking.

There are numerous types of programming languages developed in the last decades, and most of them provide interface to call C++ or C for high efficiency implementation. The motivation of Svar is to design an efficient, light-weighted and general middle-ware for multiple languages, meanwhile, brings the dynamism features from script language to C++ in a straightforward way. Firstly, a Svar class with JSON like data structure is designed to hold everything exists in C++, including basic values, functions or user defined classes and objects. Secondly, arguments are auto cast to and from Svar efficiently with compile time pointers, references and shared\_ptr detection. Thirdly, classes and functions are binded with string names to support reflection, this means all functions and classes in a shared library can be exported to a Svar object, which also calls a Svar module. The Svar modules can be accessed by different languages and this paper demonstrates how to import and use a Svar module in Python and Node.js. Moreover, the Svar modules or even a python module can also be imported by C++ at runtime, which makes C++ easier to compile and use since headers are not required anymore. We compare the performance of Svar with two state-of-the-art binding tool for Python and Node.js, and the result demonstrates that Svar is efficient, elegant and general. The core of this project is one single tiny modern C++ header with less than 5000 lines code without extra dependency. To help developers using Svar, all the source codes related are public available on http://github.com/zdzhaoyong/Svar, including documentations and benchmarks.