Multimodal image registration is the fundamental technique for scene analysis with series remote sensing images of different spectrum region. Due to the highly nonlinear radiometric relationship, it is quite challenging to find common features between images of different modal types. This paper resorts to the deep neural network, and tries to learn descriptors for multimodal image patch matching, which is the key issue of image registration. A Siamese fully convolutional network is set up and trained with a novel loss function, which adopts the strategy of maximizing the feature distance between positive and hard negative samples. The two branches of the Siamese network are connected by the convolutional operation, resulting in the similarity score between the two input image patches. The similarity score value is used, not only for correspondence point location, but also for outlier identification. A generalized workflow for deep feature based multimodal RS image registration is constructed, including the training data curation, candidate feature point generation, and outlier removal. The proposed network is tested on a variety of optical, near infrared, thermal infrared, SAR, and map images. Experiment results verify the superiority over other state-of-the-art approaches.

In this paper, we integrate the spatial-spectral information of HSI samples into non-negative matrix factorization (NMF) for affinity matrix learning to address the issue of HSI clustering. This technique consists of three main components: i) oversegmentation for computing the spectral-spatial affinity matrix, ii) NMF with the guidance of the obtained affinity matrix and iii) density-based spectral clustering on the final affinity matrix. First, the HSI is oversegmented into superpixels via the entropy rate superpixel (ERS) algorithm. The spectral-spatial affinity matrix is defined based on the class-consistency assumption of all the HSI samples in each superpixel and the similar HSI samples between adjacent superpixels. Second, to integrate the spectral-spatial information into NMF, the obtained affinity matrix is used to guide the iterative process of NMF. The spectral-spatial affinity matrix is then weighted by the affinity matrix in the obtained low-dimensional subspace to form the final affinity matrix. Third, density-based spectral clustering is applied to the final affinity matrix to obtain clustering maps. Experimental results on three public benchmark HSIs demonstrate that the proposed method is superior to the considered state-of-the-art baseline methods on both the computational cost and clustering accuracy.

Compared with general object detection with horizontal bounding boxes in natural images, oriented object detection in remote sensing images is an active and challenging research topic as objects are usually displayed in arbitrary orientations. To model the variant orientations of oriented objects, general CNN-based methods usually adopt more parameters or well-designed modules, which are often complex and inefficient. To address this issue, the detector requires two key components to deal with: (i) generating oriented proposals in a light-weight network to achieve effective representation of arbitrarily oriented objects; (ii) extracting the rotation-invariant feature map in both spatial and orientation dimensions. In this paper, we propose a novel, lightweight rotated region proposal network to produce arbitrary-oriented proposals by sliding two vertexes only on adjacent sides and adopt a simple yet effective representation to describe oriented objects. This may decrease the complexity of modeling orientation information. Meanwhile, we adopt the rotation-equivariant backbone to generate the feature map with explicit orientation channel information and utilize the spatial and orientation modules to obtain completely rotation-invariant features in both dimensions. Without tricks, extensive experiments performed on three challenging datasets DOTA-v1.0, DOTA-v1.5 and HRSC2016 demonstrate that our proposed method can reach state-of-the-art accuracy while reducing the model size by 40% in comparison with the previous best method.

Saliency, the distinctive parts of an image, has shown good potential for many applications (e.g., image interpretation and target detection). In this letter, a novel saliency detection method based on background context-aware is proposed for synthetic aperture radar (SAR) images. According to a statistical analysis of SAR image characteristics, several reference background patches (RBPs) are selected. Then, the dissimilarities between the current patch and the RBPs are used to calculate the local saliency, which is further enhanced for the final saliency under the multiscale framework. Experimental results demonstrate the effectiveness of the proposed method, which outperforms some state-of-the-art methods.

The scale parameter (SP), to control the sizes of objects, is of great significance in multiscale segmentation which is a prerequisite and foundational step for object-based change detection (OBCD). However, the appropriate SP is not readily apparent and the majority of the existing OBCD algorithms obtain the SPs by empirical or subjective trial-and-error ways that may lead to dissatisfactory accuracy and be time-consuming. To address this issue, an automatic approach for optimal segmentation scale selection for OBCD is proposed in this letter. First, a changed fuzziness image for bitemporal images was generated. Second, multiscale segmentation was implemented in a series of candidate scales, and the merging relationships between adjacent scales were built. Then, mapping the segments to the previous fuzziness image, a statistical metric describing the homogeneity of objects based on the Kullback-Leibler divergence was defined, the increments of the metric between merged objects and their child objects were calculated and weighted to identify the optimal SPs. For performance evaluation, Dempter-Shafer (DS) evidence fusion was utilized in the scales selected by the proposed approach in comparison with other state of art or empirical ones. The experimental results employing GF-1, Google Earth, and aerial images demonstrated the superiority and effectiveness of the SPs identified by the proposed approach in the OBCD task.

The very high spatial resolution (VHR) remote sensing images have been an extremely valuable source for monitoring changes occurring on the Earth’s surface. However, precisely detecting relevant changes in VHR images still remains a challenge, due to the complexity of the relationships among ground objects. To address this limitation, a dual neighborhood hypergraph neural network is proposed in this article, which combines multiscale superpixel segmentation and hypergraph convolution to model and exploit the complex relationships. First, the bi-temporal image pairs are segmented under two scales and fed to a pre-trained U-net to obtain node features by treating each object under the fine scale as a node. The dual neighborhood is then defined using the father-child and adjacent relationships of the segmented objects to construct the hypergraph, which permits models to represent higher-order structured information far more complex than the conventional pairwise relationships. The hypergraph convolutions are conducted on the constructed hypergraph to propagate the label information from a small amount of labeled nodes to the other unlabeled ones by the node-edge-node transformation. Moreover, to alleviate the problem of imbalanced sampling, the focal loss function is adopted to train the hypergraph neural network. The experimental results on optical, SAR and heterogeneous optical/SAR data sets demonstrate that the proposed method offersbetter effectiveness and robustness compared to many state-of-the-art methods.