Fourier-based image correlation is a powerful area-based image registration technique, which involves aligning images based on a translation model or similarity model by means of the image information and operation in the frequency domain. In recent years, Fourier-based image correlation has made significant progress and attracted extensive research interest in a variety of applications, especially in the field of photogrammetry and remote sensing, leading to the development of a number of subpixel methods that have improved the accuracy and robustness. However, to date, a detailed review of the literature related to Fourier-based image correlation is still lacking. In this review, we aim at providing a comprehensive overview of the fundamentals, developments, and applications of image registration with Fourier-based image correlation methods. Specifically, this review introduces the principal laws underlying these methods, presents a survey of the existing subpixel methods calculated both in the spatial domain and in the frequency domain, summarizes the major applications from three aspects, and discusses the challenges and possible directions of future research. This review is expected to be beneficial for researchers working in the relevant fields to obtain an insight into the current state of the art, to develop new variants, to explore potential applications, and to suggest promising future trends of image registration with Fourier-based image correlation.

Three-dimensional (3D) digital technology is essential to the maintenance and monitoring of cultural heritage sites. In the field of bridge engineering, 3D models generated from point clouds of existing bridges is drawing increasing attention. Currently, the widespread use of the unmanned aerial vehicle (UAV) provides a practical solution for generating 3D point clouds as well as models, which can drastically reduce the manual effort and cost involved. In this study, we present a semi-automated framework for generating structural surface models of heritage bridges. To be specific, we propose to tackle this challenge via a novel top-down method for segmenting main bridge components, combined with rule-based classification, to produce labeled 3D models from UAV photogrammetric point clouds. The point clouds of the heritage bridge are generated from the captured UAV images through the structure-from-motion workflow. A segmentation method is developed based on the supervoxel structure and global graph optimization, which can effectively separate bridge components based on geometric features. Then, recognition by the use of a classification tree and bridge geometry is utilized to recognize different structural elements from the obtained segments. Finally, surface modeling is conducted to generate surface models of the recognized elements. Experiments using two bridges in China demonstrate the potential of the presented structural model reconstruction method using UAV photogrammetry and point cloud processing in 3D digital documentation of heritage bridges. By using given markers, the reconstruction error of point clouds can be as small as 0.4%. Moreover, the precision and recall of segmentation results using testing date are better than 0.8, and a recognition accuracy better than 0.8 is achieved.

The semantic labeling of the urban area is an essential but challenging task for a wide variety of applications such as mapping, navigation, and monitoring. The rapid advance in Light Detection and Ranging (LiDAR) systems provides this task with a possible solution using 3D point clouds, which are accessible, affordable, accurate, and applicable. Among all types of platforms, the airborne platform with LiDAR can serve as an efficient and effective tool for large-scale 3D mapping in the urban area. Against this background, a large number of algorithms and methods have been developed to fully explore the potential of 3D point clouds. However, the creation of publicly accessible large-scale annotated datasets, which are critical for assessing the performance of the developed algorithms and methods, is still at an early age. In this work, we present a large-scale aerial LiDAR point cloud dataset acquired in a highly-dense and complex urban area for the evaluation of semantic labeling methods. This dataset covers an urban area with highly-dense buildings of approximately 1 km² and includes more than three million points with five classes of objects labeled. Moreover, experiments are carried out with the results from several baseline methods, demonstrating the feasibility and capability of the dataset serving as a benchmark for assessing semantic labeling methods.

Attitude jitter is a common phenomenon in the application of high resolution satellites, which may result in large errors of geo-positioning and mapping accuracy. Therefore, it is critical to detect and compensate attitude jitter to explore the full geometric potential of high resolution satellites. In this paper, a framework of jitter detection and compensation for high resolution satellites is proposed and some preliminary investigation is performed. Three methods for jitter detection are presented as follows. (1) The first one is based on multispectral images using parallax between two different bands in the image; (2) The second is based on stereo images using rational polynomial coefficients (RPCs); (3) The third is based on panchromatic images employing orthorectification processing. Based on the calculated parallax maps, the frequency and amplitude of the detected jitter are obtained. Subsequently, two approaches for jitter compensation are conducted. (1) The first one is to conduct the compensation on image, which uses the derived parallax observations for resampling; (2) The second is to conduct the compensation on attitude data, which treats the influence of jitter on attitude as correction of charge-coupled device (CCD) viewing angles. Experiments with images from several satellites, such as ASTER (Advanced Spaceborne Thermal Emission and Reflection Radiaometer), LRO (Lunar Reconnaissance Orbiter) and ZY-3 (ZiYuan-3) demonstrate the promising performance and feasibility of the proposed framework.

Matching multi-modal remote sensing images (MRSI) is a challenging task. Due to significant nonlinear radiation differences (NRD), traditional image matching methods cannot achieve satisfactory results. Current research shows that structural information can get more robust matching results compared with texture information (i.e., gradient features) from images. In order to better explore the structural information of images, this paper proposes a MRSIs matching method using structure saliency features, called weighted structure saliency feature (WSSF). Two strategies are investigated and integrated in WSSF to improve the matching performance. The scale space is constructed based on the pointwise shape-adaptive texture scale filtering, which can better retain the structure features, and the second-order Gaussian steerable filtering, edge confidence map and phase features are combined to establish the structural saliency map combined with second-order Gaussian steerable filtering, which is much more robust to NRD than traditional gradient map. The performance of the proposed method was evaluated on a total of 120 image pairs from two MRSI datasets, and compared with state-of-the-art matching methods including the histogram of the orientation of weighted phase (HOWP), locally normalized image feature transform (LNIFT), co-occurrence filter space matching (CoFSM), radiation-variation insensitive feature transform (RIFT), local phase sharpness orientation (LPSO), and position-scale-orientation SIFT (PSO-SIFT). Experimental results indicate that WSSF obtains satisfactory and reliable results in terms of success rate and matching accuracy. Compared with the above six methods, the matching accuracy of WSSF is improved by more than 20.275%, and the success rate is improved by over 5.833%. The source code will be publicly available in https://github.com/WGY-RS/WSSF.