Although the scale-invariant feature transform (SIFT) algorithm has been successfully applied to both optical image registration and synthetic aperture radar (SAR) image registration, SIFT-like algorithms have failed to register high-resolution (HR) optical and SAR images due to large geometric differences and intensity differences. In this paper, to perform optical-to-SAR (OS) image registration, we proposed an advanced SIFT-like algorithm (OS-SIFT) that consists of three main modules: keypoint detection in two Harris scale spaces, orientation assignment and descriptor extraction, and keypoint matching. Considering the inherent properties of SAR images and optical images, the multiscale ratio of exponentially weighted averages and multiscale Sobel operators are used to calculate consistent gradients for the SAR images and optical images on the basis of which, as a result, two Harris scale spaces can be constructed. Keypoints are detected by finding the local maxima in the scale space followed by a localization refinement method based on the spatial relationship of the keypoints. Moreover, gradient location orientation histogram-like descriptors are extracted using multiple image patches to increase the distinctiveness. The experimental results on simulated images and several HR satellite images show that the proposed OS-SIFT algorithm gives a robust registration result for optical-to-SAR images and outperforms other state-of-the-art algorithms in terms of registration accuracy.

In this paper, we propose an automatic and efficient method to solve optical and SAR image registration using the improved phase congruency (PC) model. First, evenly-distributed keypoints are extracted from the optical images via the block harris method. Complementary grid points are then selected in image regions with poor structural information and supplemented to the keypoint set. For each keypoint, a robust feature representation that captures the local spatial relationship is proposed based on the improved PC model. Specifically, we propose to use two different PC models, the classic PC and the SAR-PC, to construct features for optical and SAR images respectively. The PC features of several directions are aggregated to construct the feature descriptors, and a similarity metric via the phase correlation of feature descriptors is obtained. The proposed similarity metric can not only find accurate correspondence but also present efficient results without presetting the size of the search region. We compare the proposed method with two baselines and state-of-the-art (SOTA) methods, i.e. OS-SIFT, HOPC, CFOG, in various scenarios. The results show that the proposed method outperforms the baselines and shows comparable performance with SOTA methods in regions with abundant structural information and better performance in regions with less structural information. Moreover, we build a high-resolution optical and SAR image matching dataset, which consists of 10,692 non-overlapping patch-pairs of 256x256 pixels and 1-m resolution. Results of two benchmarks, Siamese deep matching network and conditional Generative Adversarial Networks, show that this dataset is practical and challenging.

This letter proposes a method for the change detection in multisensor remote sensing images. The proposed method combines multitemporal segmentation and compound classification. In consideration of the particularity of multisensor images, multitemporal segmentation is applied to generate homogeneous objects. This process can reduce the salt and pepper effect that is inevitable in pixel-based methods and reduce the false alarms caused by area transitions and object misalignment in traditional object-based methods. Then, compound classification is carried out at the object level. This process exploits temporal correlations and overcomes the error propagation of traditional postclassification comparison methods. The change map is generated by comparing the classification maps at different times. Experimental validation is conducted with GaoFen3, Terrasar, GaoFen2, and Google Earth data.

Phase correlation (PC), an efficient frequency-domain registration method, has been extensively used in remote sensing images owing to its subpixel accuracy and robustness to image contrast, noise, and occlusions. However, its performance becomes poor when applied to the registration between optical and synthetic aperture radar (SAR) images, which are two typical multisensor images. Inspired by the recently proposed feature-based methods, we present a novel subpixel registration method that combines robust feature representations of optical and SAR images and the 3-D PC (OS-PC). The robust feature representations, which capture the inherent property of the two images and retain their structural information, form two dense image cubes. The 3-D PC utilizes the image cubes as a substitute of two raw images to estimate 2-D translations, either by locating peak in the spatial domain or by directly working in the Fourier domain. Furthermore, we investigate two techniques to improve the accuracy of the 3-D PC both in the spatial domain and Fourier domain: the first is the constrained energy minimization method to seek the Dirac delta function after 3-D inverse Fourier transform and the second is the fast sample consensus fitting to estimate phase difference after high-order singular value decomposition of the PC matrix. Experiments with both simulated and satellite optical-to-SAR pairs were carried out to test the proposed method. Compared with state-of-the-art PC methods and optical-to-SAR registration methods, the proposed method presents a superior performance in both accuracy and robustness. Moreover, we verify the adaptability of the proposed method.

Automatic building extraction from high-resolution remote sensing images has many practical applications, such as urban planning and supervision. However, fine details and various scales of building structures in high-resolution images bring new challenges to building extraction. An increasing number of neural network-based models have been proposed to handle these issues, while they are not efficient enough, and still suffer from the error ground truth labels. To this end, we propose an efficient end-to-end model, EU-Net, in this paper. We first design the dense spatial pyramid pooling (DSPP) to extract dense and multi-scale features simultaneously, which facilitate the extraction of buildings at all scales. Then, the focal loss is used in reverse to suppress the impact of the error labels in ground truth, making the training stage more stable. To assess the universality of the proposed model, we tested it on three public aerial remote sensing datasets: WHU aerial imagery dataset, Massachusetts buildings dataset, and Inria aerial image labeling dataset. Experimental results show that the proposed EU-Net is superior to the state-of-the-art models of all three datasets and increases the prediction efficiency by two to four times.

As two of the most widely used remote sensing images, optical and synthetic aperture radar (SAR) images show abundant and complementary information on the same target owing to their individual imaging mechanisms. Consequently, using optical and SAR images simultaneously can better describe the inherent features of the target, and thus, be beneficial for subsequent remote sensing applications. In this article, we propose a novel modular fully convolutional network model to improve the accuracy of land cover classification by fully exploiting the complementary features of the two sensors. We investigate where and how to fuse the two images in the joint classification network. A cross-gate module with a bidirectional information flow is proposed to achieve the best fusion performance. In addition, to validate the proposed model, we construct a multiclass land cover classification dataset. Exhaustive experiments show that the proposed joint classification network presents superior results than state-of-the-art classification models using single-sensor images.