The matching problem for heterologous remote sensing images can be simplified to the matching problem for pseudo homologous remote sensing images via image translation to improve the matching performance. Among such applications, the translation of synthetic aperture radar (SAR) and optical images is the current focus of research. However, the existing methods for SAR-to-optical translation have two main drawbacks. First, single generators usually sacrifice either structure or texture features to balance the model performance and complexity, which often results in textural or structural distortion; second, due to large nonlinear radiation distortions (NRDs) in SAR images, there are still visual differences between the pseudo-optical images generated by current generative adversarial networks (GANs) and real optical images. Therefore, we propose a dual-generator translation network for fusing structure and texture features. On the one hand, the proposed network has dual generators, a texture generator, and a structure generator, with good cross-coupling to obtain high-accuracy structure and texture features; on the other hand, frequency-domain and spatial-domain loss functions are introduced to reduce the differences between pseudo-optical images and real optical images. Extensive quantitative and qualitative experiments show that our method achieves state-of-the-art performance on publicly available optical and SAR datasets. Our method improves the peak signal-to-noise ratio (PSNR) by 21.0%, the chromatic feature similarity (FSIMc) by 6.9%, and the structural similarity (SSIM) by 161.7% in terms of the average metric values on all test images compared with the next best results. In addition, we present a before-and-after translation comparison experiment to show that our method improves the average keypoint repeatability by approximately 111.7% and the matching accuracy by approximately 5.25%.

There are large nonlinear radiance differences between optical and synthetic aperture radar (SAR) from different imaging sensors. When combined with rotational and scale variations, achieving equivariant matching becomes highly challenging. The existing single network models face two primary limitations, insufficient learning of modal invariant features and insufficient characterization of rotation and scale. After task decomposition, we propose a novel rotation and scale equivariant network (RSENet) for optical-SAR image matching, comprising two modules, SAR-to-optical image translation module and equivariant descriptors (ED). First, we propose an image translation network to obtain more beneficial pseudo-optical images for matching. Second, we achieve high-precision matching via equivariant descriptors designed through scale angle estimation (SAE) and descriptor colearning. The network initially corrects image blocks explicitly via scale and angle loss functions that are based on periodic encoding (PE) learning. Subsequently, it employs an attention module based on cross-scale perception and feature channel excitation to extract hierarchical information from the corrected image blocks and to enhance the local feature representation of the descriptors. We validate the effectiveness of RSENet by conducting numerous experiments under combined transformations of various angles and scales. Our proposed method achieves an approximately 28% improvement in the matching success rate compared with existing methods.

Optical and synthetic aperture radar (SAR) image keypoint detection is an important foundation for multimodal remote sensing image matching. The influence of nonlinear radiometric differences and geometric deformation between optical and SAR images leads to low repeatability of existing keypoint detection methods. To address the problem that existing keypoint detection methods cannot provide the required homonymous points for heterogenous image matching, we propose a keypoint detection method (SKD-Net) for optical and SAR images, and improve it in terms of both network structure and network optimization. First, we propose a multilevel attention Siamese network, which is composed of multiple convolutional modules and transformer modules with shared weights to extract common features at different levels for keypoint detection. We introduce a transformer module in the keypoint detection pipeline and fuse shallow and deep features to obtain more spatial and rich semantic information to facilitate heterogeneous image keypoint detection. Then, to ensure that the detected keypoints have more homonymous points and localization accuracy, we propose a position consistent loss. Unlike previous loss functions, our designed position-consistent loss function takes the differences between heterogeneous image score maps into account, and it autonomously selects the optimized correct point pairs to enable the network to perform correct learning. Finally, extensive experiments show that our detection method outperforms the current state-of-the-art keypoint detection methods in terms of repeatability, localization accuracy, and matching performance. Our source code is available at https://github.com/zhangschen/ SKD-Net.