Deep learning-based change detection (CD) using remote sensing images has received increasing attention in recent years. However, how to effectively extract and fuse the deep features of bi-temporal images for improving the accuracy of CD is still a challenge. To address that, a novel adjacent-level feature fusion network with 3D convolution (named AFCF3D-Net) is proposed in this article. First, through the inner fusion property of 3D convolution, we design a new feature fusion way that can simultaneously extract and fuse the feature information from bi-temporal images. Then, to alleviate the semantic gap between low-level features and high-level features, we propose an adjacent-level feature cross-fusion (AFCF) module to aggregate complementary feature information between the adjacent levels. Furthermore, the full-scale skip connection strategy is introduced to improve the capability of pixel-wise prediction and the compactness of changed objects in the results. Finally, the proposed AFCF3D-Net has been validated on the three challenging remote sensing CD datasets: the Wuhan building dataset (WHU-CD), the LEVIR building dataset (LEVIR-CD), and the Sun Yat-Sen University dataset (SYSU-CD). The results of quantitative analysis and qualitative comparison demonstrate that the proposed AFCF3D-Net achieves better performance compared to other state-of-the-art methods. The code for this work is available at https://github.com/wm-Githuber/AFCF3D-Net.

Identifying feature correspondences between multimodal images is facing enormous challenges because of the significant differences both in radiation and geometry. To address these problems, we propose a novel feature matching method (named R ₂ FD ₂ ) that is robust to radiation and rotation differences, which consists of a repeatable feature detector and a rotation-invariant feature descriptor. In the first stage, a repeatable feature detector called the Multi-channel Auto-correlation of the Log-Gabor (MALG) is presented for feature detection, which combines the multi-channel auto-correlation strategy with the Log-Gabor wavelets to detect interest points (IPs) with high repeatability and uniform distribution. In the second stage, a rotation-invariant feature descriptor is constructed, named the Rotation-invariant Maximum index map of the Log-Gabor (RMLG), which includes fast assignment of dominant orientation and construction of feature representation. In the process of fast assignment of dominant orientation, a Rotation-invariant Maximum Index Map (RMIM) is built to address rotation deformations. Then, the proposed RMLG incorporates the rotation-invariant RMIM with the spatial configuration of DAISY to improve RMLG’s resistance to radiation and rotation variances. Finally, we conduct experiments to validate the matching performance of our R ₂ FD ₂ utilizing different types of multimodal image datasets. Experimental results show that the proposed R ₂ FD ₂ outperforms five state-of-the-art feature matching methods. Moreover, our R ₂ FD ₂ achieves the accuracy of matching within two pixels and has a great advantage in matching efficiency over contrastive methods.

With the expansion of optical and SAR image fusion application scenarios, it is necessary to integrate their information in land classification, feature recognition, and target tracking. Current methods focus excessively on integrating multimodal feature information to enhance the information richness of the fused images, whereas neglecting the highly corrupted visual perception of the fused results by modal differences and SAR speckle noise. To address that, this paper proposes a novel optical and SAR image fusion framework named Visual Saliency Features Fusion (VSFF), which is based on the extraction and balancing of significant complementary features of optical and SAR images. Firstly, we propose a decomposition algorithm of complementary features to divide the image into main structure features and detail texture features. Then, for the fusion of main structure features, we reconstruct the visual saliency features maps of the pixel and structure that contain significant information from optical and SAR images, and input them into a total variation constraint model to compute the fusion result and achieve the optimal information transfer. Meanwhile, we construct a new feature descriptor based on the Gabor wavelet, that separates meaningful detail texture features from residual noise and selectively preserves features that can improve the interpretability of fusion result. In a comparative analysis with seven state-of-the-art fusion algorithms, VSFF achieved better results in qualitative and quantitative evaluations, and our fused images have a clear and appropriate visual perception. The source code is publicly available at https://github.com/yeyuanxin110/VSFF.

Automatic registration of synthetic aperture radar (SAR) and optical images is still a challenging problem because of the potential differences in geometry and intensity. In this work, we propose a robust and efficient method for improving the registration performance of SAR and optical images. Our work consists mainly of two steps, including the feature point detection stage and the feature description stage. In the first stage, we present a new feature extraction method, named the nonlinear diffusion-based Harris–Laplace (NDHL) detector, which incorporates the nonlinear diffusion and a spatial consistency strategy into the Harris–Laplace detector, which can weaken the modality variations, enhance the structural features of the multimodal images, and can detect many more similar and highly repeatable feature points. In the second stage, we design a novel structural descriptor, named multiscale adaptive binning phase congruency (MABPC). The proposed MABPC descriptor encodes multiscale phase congruency features with an adaptive binning spatial structure, which brings an improvement of the robustness against geometric and nonlinear intensity discrepancies. Experimental results on both simulated and real image pairs show that the proposed registration method achieves encouraging performance improvements over other state-of-the-art methods. In comparison with the robust optical and SAR image registration based on phase congruency (ROS-PC) and radiation-variation insensitive feature transform (RIFT) methods, the proposed method obtains an average 54.6% improvement for the number of correct matches and has an average registration precision of 1.38 pixels.

Deep learning (DL) techniques for change detection have undergone rapid development in the last few years. However, it is still a challenge how to reduce massive network parameters and sufficiently fuse bi-temporal image features to improve detection accuracy. Therefore, this work proposes a novel and lightweight network based on feature interleaved fusion and bi-stage decoding (FFBDNet) for change detection. In the encoding stage, considering the application problems caused by a large number of network parameters, we use the more efficient EfficientNet as the backbone to extract the bi-temporal image features based on Siamese architecture. To fuse the bi-temporal image features and reduce interference from surrounding objects, we propose a feature interleaved fusion module (FIFM), which can interleave the shared feature information and the difference variance feature information. During the decoding stage, the fused features are split into two groups, and a novel bi-stage decoding framework is proposed to generate the accuracy change map gradually. Extensive experiments and ablation studies are validated on three public change detection datasets: the WHU-CD, LEVIR-CD, and SYSU-CD datasets. Compared to state-of-the-art (SOTA) methods, the experimental results demonstrate that the proposed FFBDNet produces a better balance between performance and model parameters. Specifically, the F1 values obtained for these three datasets are 93.27%, 91.11%, and 80.10%, respectively, and the model parameters of the network are just 2.85 M.

The structural features using self-similarity have become more popular for multimodal remote sensing image matching. However, mostly because of significant geometric distortions and nonlinear intensity differences between images, these methods produce a limited matching performance when directly applied to multimodal remote sensing images. To address that, we propose a novel feature descriptor named pyramid features of orientated self-similarity (POSS) for multimodal remote sensing image matching, which integrates phase congruency into the self-similarity model for better encoding structural information. Unlike these conventional self-similarity-based descriptors, the POSS is constructed by using phase congruency instead of image intensity in a pyramid manner and thus has a better discriminability and robustness to modality variations. Additionally, we design a uniform multi-moment of phase congruency (UMMPC) detector for feature detection, which can improve the number, distribution, and repeatability of feature points. Experimental results demonstrate its superior applicability of the proposed method over the state-of-the-art methods, showing much better matching performance.