Multimodal image registration is the fundamental technique for scene analysis with series remote sensing images of different spectrum region. Due to the highly nonlinear radiometric relationship, it is quite challenging to find common features between images of different modal types. This paper resorts to the deep neural network, and tries to learn descriptors for multimodal image patch matching, which is the key issue of image registration. A Siamese fully convolutional network is set up and trained with a novel loss function, which adopts the strategy of maximizing the feature distance between positive and hard negative samples. The two branches of the Siamese network are connected by the convolutional operation, resulting in the similarity score between the two input image patches. The similarity score value is used, not only for correspondence point location, but also for outlier identification. A generalized workflow for deep feature based multimodal RS image registration is constructed, including the training data curation, candidate feature point generation, and outlier removal. The proposed network is tested on a variety of optical, near infrared, thermal infrared, SAR, and map images. Experiment results verify the superiority over other state-of-the-art approaches.

Detecting shallow buried antipersonnel mines (APMs) with a ground-penetrating radar (GPR) is a challenging task because of clutter contamination, which often obscures the APM response. In this letter, a novel method combining migration imaging with the low-rank and sparse representation method to suppress clutter and extract target image is presented. The proposed method first focuses and strengthens the target response with migration imaging. Then, since the focused target response and clutter, respectively, constitute the sparse component and the low-rank component of the recorded data, the recently proposed robust principal component analysis (RPCA) can be applied to the recorded data to separate the target response (sparse component) from the clutter (low-rank component). Numerical simulation and experiments with real GPR systems are conducted. Results demonstrate the effectiveness of the proposed method in improving signal-to-clutter ratio and retrieving geometrical information of the target, which permits a better APM identification in heavy clutter environment.

In existing superpixel-wise segmentation algorithms, superpixel generation most often is an isolated preprocessing step. The segmentation performance is determined to a certain extent by the accuracy of superpixels. However, it is still a challenge to develop a stable superpixel generation method. In this article, we attempt to incorporate the superpixel generation and merging steps into an end-to-end trainable deep network. First, we employ a recently proposed differentiable superpixel generation method to over-segment the single-polarization synthetic aperture radar (SAR) image. It outputs the statistical likelihood that each pixel belongs to different superpixels. In superpixel merging part, as one of our main contributions, we propose a superpixel-wise statistical dissimilarity measure method for converting the soft superpixels set into a self-connected weighted graph. More importantly, inspired by the concept of the number of walks in graph theory, we define the $k$ -order connectivity of each vertex. This definition can intelligently indicate the potential soft cluster centers and class assignments in graph. This merging method is differentiable, computationally simple, and free of empirical parameters. The superpixel generation and merging phases can be implemented under a unified deep network. The benefit is that our method can iteratively adjust the shapes of the superpixels according to the boundaries and segmentation results during training, until the satisfactory segmentation results are captured. Experimental results on real SAR images demonstrate that the segmentation precision of our proposed method is superior to other state-of-the-art methods in terms of precision and computational efficiency.

Since the number of superpixels is lower than that of pixels, superpixels can substantially speed up subsequent processing steps and have been widely used in synthetic aperture radar (SAR) image segmentation. However, in most of the existing superpixel-wise segmentation algorithms, superpixel prediction is an isolated preprocessing step and is independent of the segmentation task. The performance of the segmentation results is determined by the accuracy of superpixels. Once superpixels are generated, their shape cannot be changed in the following segmentation stage, even if the same superpixels contain pixels of different landcovers. To address this, we propose an end-to-end trainable superpixel-wise segmentation method for single-polarization SAR images. First, we design a differentiable boundary-ware clustering method for estimating task-specific superpixels. Instead of the hard association between pixels and superpixels in the existing superpixel algorithms, this method introduces the soft association map to make the clustering differentiable. Hence, it can be implemented using a simple deep fully convolutional network. In the segmentation part, we propose a novel soft graph convolution network (Soft-GCN), which takes the association map as input and performs superpixel-wise segmentation. The advantage of our method is that superpixel generation and graph convolution parts can be trained under a unified framework, until two parts obtain the optimum parameters. In the training process, it can adaptively adjust the shape of the superpixels according to the segmentation results, ensuring the superpixels correctly adhere the boundaries. Experimental results with simulated and real SAR images demonstrate that our method outperforms other state-of-the-art segmentation algorithms, while also being faster.

Simple linear iterative clustering (SLIC) algorithm was proposed for superpixel generation on optical images and showed promising performance. Several studies have been proposed to modify SLIC to make it applicable for polarimetric synthetic aperture radar (PolSAR) images, where the Wishart distance is adopted as the similarity measure. However, the superpixel segmentation results of these methods were not satisfactory in heterogeneous urban areas. Further, it is difficult to determine the tradeoff factor which controls the relative weight between polarimetric similarity and spatial proximity. In this research, an adaptive polarimetric SLIC (Pol-ASLIC) superpixel generation method is proposed to overcome these limitations. First, the spherically invariant random vector (SIRV) product model is adopted to estimate the normalized covariance matrix and texture for each pixel. A new edge detector is then utilized to extract PolSAR image edges for the initialization of central seeds. In the local iterative clustering, multiple cues including polarimetric, texture, and spatial information are considered to define the similarity measure. Moreover, a polarimetric homogeneity measurement is used to automatically determine the tradeoff factor, which can vary from homogeneous areas to heterogeneous areas. Finally, the SLIC superpixel generation scheme is applied to the airborne Experimental SAR and PiSAR L-band PolSAR data to demonstrate the effectiveness of this proposed superpixel generation approach. This proposed algorithm produces compact superpixels which can well adhere to image boundaries in both natural and urban areas. The detail information in heterogeneous areas can be well preserved.

Automatic registration of optical and synthetic aperture radar (SAR) images is one of the most challenging tasks due to the influence of speckle noise and nonlinear radiation differences. In this article, we propose a compound registration method for optical and SAR images based on feature decoupling network (FDNet), which consists of a residual denoising network (RDNet) and a pseudo-Siamese fully convolutional network (PSFCN). First, we propose a fast compound matching algorithm, which can overcome the respective weaknesses of the registration accuracy and computational complexity of the feature-based and area-based methods. Specifically, FAST keypoint detection is used to generate the center of the initial template. The extraction of local feature descriptors and the matching of the initial templates are implemented by PSFCN. Second, we design an RDNet to learn the statistical model of speckle noise in SAR images and define a new loss function based on mean-square error (mse) and total variation (TV) to achieve the propagation of speckle noise. PSFCN and RDNet are used to learn deep representations of semantic and noise information, respectively. Then, the semantic and noise features are decoupled on spaced convolutional layers. Finally, the optimal matching templates are searched in a small search window around the initial matching templates. In addition, we propose a strategy for adaptively selecting the template size based on 2-D entropy, which can select the appropriate template size according to the content richness of SAR images. Registration results on a public registration dataset show that our proposed method achieves better performance than other state-of-the-art methods.