Fourier-based image correlation is a powerful area-based image registration technique, which involves aligning images based on a translation model or similarity model by means of the image information and operation in the frequency domain. In recent years, Fourier-based image correlation has made significant progress and attracted extensive research interest in a variety of applications, especially in the field of photogrammetry and remote sensing, leading to the development of a number of subpixel methods that have improved the accuracy and robustness. However, to date, a detailed review of the literature related to Fourier-based image correlation is still lacking. In this review, we aim at providing a comprehensive overview of the fundamentals, developments, and applications of image registration with Fourier-based image correlation methods. Specifically, this review introduces the principal laws underlying these methods, presents a survey of the existing subpixel methods calculated both in the spatial domain and in the frequency domain, summarizes the major applications from three aspects, and discusses the challenges and possible directions of future research. This review is expected to be beneficial for researchers working in the relevant fields to obtain an insight into the current state of the art, to develop new variants, to explore potential applications, and to suggest promising future trends of image registration with Fourier-based image correlation.

Compared with traditional manned airborne photogrammetry, unmanned aerial vehicle remote sensing (UAVRS) has the advantages of lower cost and higher flexibility in data acquisition. It has, therefore, found various applications in fields such as three-dimensional (3D) mapping, emergency management, and so on. However, due to the instability of the UAVRS platforms and the low accuracy of the onboard exterior orientation (EO) observations, the use of direct georeferencing image data leads to large location errors. Light detection and ranging (LiDAR) data, which is highly accurate 3D information, is treated as a complementary data source to the optical images. This paper presents a semi-automatic approach for the registration of UAVRS images and airborne LiDAR data based on linear control features. The presented approach consists of three main components, as follows. (1) Buildings are first separated from the point cloud by the integrated use of height and size filtering and RANdom SAmple Consensus (RANSAC) plane fitting, and the 3D line segments of the building ridges and boundaries are semi-automatically extracted through plane intersection and boundary regularization with manual selections; (2) the 3D line segments are projected to the image space using the initial EO parameters to obtain the approximate locations, and all the corresponding 2D line segments are semi-automatically extracted from the UAVRS images. Meanwhile, the tie points of the UAVRS images are generated using a Förstner operator and least-squares image matching; and (3) by use of the equations derived from the coplanarity constraints of the linear control features and the colinear constraints of the tie points, block bundle adjustment is carried out to update the EO parameters of the UAVRS images in the coordinate framework of the LiDAR data, achieving the co-registration of the two datasets. Experiments were performed to demonstrate the validity and effectiveness of the presented method, and a comparison with the traditional registration method based on LiDAR intensity images showed that the presented method is more accurate, and a sub-pixel accuracy level can be achieved.

The Fourier-based image correlation technique has been widely concerned due to its accuracy, efficiency, and robustness to image contrast and brightness. Accordingly, a variety of subpixel methods have been proposed. However, the detailed subpixel-level influence of the complicated radiometric variations has yet to be investigated, and few corresponding improvements have been made. This paper presents a novel illumination-robust subpixel Fourier-based image correlation method based on phase congruency. Both the magnitude and orientation information of the phase congruency features are adopted to construct a structural image representation. The image representation is then embedded into the correlation scheme of the subpixel methods, either by linear phase estimation in the frequency domain or by kernel fitting in the spatial domain, achieving two improved subpixel methods. The proposed methods integrate the advantages of the structural image representation and the original correlation scheme, and make full use of both global and local phase information to achieve illumination-robust correlation. Experiments undertaken with both simulated and real radiometric differences were carried out with ground-truth subpixel shifts. The performances of the proposed methods and the other state-of-the-art subpixel Fourier-based correlation methods were evaluated and compared. The experimental results indicate that the proposed methods outperform the other methods in the presence of diverse radiometric variations, in both accuracy and robustness.

Spaceborne laser altimetry technology assists global DEMs to improve the accuracy of elevation data due to its highly accurate range and wide coverage. As compared to the previous laser altimeter systems used for Earth observation, ICESat-2 has a sensitivity for photon detection that can provide more accurate and denser surface elevation observations. This paper proposed a DEM correction model using ICESat-2 data. The model used the altimetric data to verify the DEM elevation errors in ICESat-2 coverage areas firstly. Then an attribute set was constructed to evaluate the error sources of the global-scale DEM. The evaluations of the error sources include the location/positioning of the platform, atmospheric conditions, topographic relief, land cover, and heterologous infill data, etc. Finally, a regression model was constructed by the attribute set and the DEM elevation errors within ICESat-2 coverage areas, in order to correct the DEM in areas without ICESat-2. In the validation experiments, this study conducted elevation correction experiments using the ASTER Global Digital Elevation Model (GDEM) and the Shuttle Radar Topography Mission (SRTM) in three regions around the world and applied the airborne LiDAR data in each region to verify the corrected results. The results showed that the proposed model was suitable for the elevation correction of global-scale DEMs and can be applied to more than 90% of global land, i.e., land areas with a slope less than 25°. The accuracy improvement ratios of the corrected GDEM were 17.89%–33.33% across the different types of topography, and the accuracy improvement ratios of the corrected SRTM were 27.77%–44.64% across different types of topography.

Satellite platform vibration induced by the onboard dynamic components and exterior perturbation deteriorates platform stability and causes attitude jitter, resulting in image distortion and geometric accuracy degradation. This paper presents an attitude jitter detection method based on images and dense ground controls, which requires neither high-performance attitude measurement devices nor specific sensor configuration like parallax observation. Attitude variations will result in image space discrepancies at control points, from which the attitude jitter can be retrieved. The method was validated by a case study with ZY-3 satellite, which is the first civilian high-resolution stereo mapping satellite in China. The experimental results show that an almost constant jitter frequency of about 0.65 Hz was detected, which was consistent with the direct attitude observations. The photogrammetric method is theoretically capable of detecting attitude jitter up to half of the image line scanning rate according to Shannon's sampling theorem, which is far beyond the detectable range by direct attitude observations at a frequency usually not higher than 4 Hz. Currently, the attitude jitter of the roll angle and the pitch angle can be effectively detected and accurately estimated for the nadir camera. The pitch angle jitter is also well estimable for the forward and the backward cameras. With approximation, the roll angle jitter for the two off-nadir cameras can also be estimated, though with some deviation caused by the influence of the yaw angle jitter.

Ice, Cloud, and land Elevation Satellite-2 (ICESat-2), the first photon-counting laser altimetry satellite, is the most advanced on-orbit altimetry system in the world. The data obtained by it contain a large number of background photons, limited by sensitive photon detection system. In this study, a density-based adaptive method (DBAM) for photons detection of ground and canopy method is proposed, which is aimed at solving the problem of signal photons detection in vegetation areas. First, the photon density is homogenized according to the noise photon rate, to reduce the effect of uneven background noise. Then, the ground signal photons were extracted by the search ellipse adaptively changing direction and size along the slope direction to find the maximum density direction. The canopy signal photons were extracted again from the rest photons of the first step by using a vertical elliptical search area. At last of the DBAM, the photons between the ground and the canopy are extracted as vegetation signal photons. The effectiveness of DBAM is evaluated both quantitatively and qualitatively. The results show that the proposed method can effectively detect ground and canopy photons, with the Kappa coefficient of 0.88 and 0.91 of two selected datasets. Compared to the results of ATL08, DBAM can better adapt to the slope, the extracted ground photons have better continuity and can extract more accuracy vegetation photons.