Automatic registration of multimodal remote sensing data [e.g., optical, light detection and ranging (LiDAR), and synthetic aperture radar (SAR)] is a challenging task due to the significant nonlinear radiometric differences between these data. To address this problem, this paper proposes a novel feature descriptor named the histogram of orientated phase congruency (HOPC), which is based on the structural properties of images. Furthermore, a similarity metric named HOPCncc is defined, which uses the normalized correlation coefficient (NCC) of the HOPC descriptors for multimodal registration. In the definition of the proposed similarity metric, we first extend the phase congruency model to generate its orientation representation and use the extended model to build HOPCncc. Then, a fast template matching scheme for this metric is designed to detect the control points between images. The proposed HOPCncc aims to capture the structural similarity between images and has been tested with a variety of optical, LiDAR, SAR, and map data. The results show that HOPCncc is robust against complex nonlinear radiometric differences and outperforms the state-of-the-art similarities metrics (i.e., NCC and mutual information) in matching performance. Moreover, a robust registration method is also proposed in this paper based on HOPCncc, which is evaluated using six pairs of multimodal remote sensing images. The experimental results demonstrate the effectiveness of the proposed method for multimodal image registration.

Although image matching techniques have been developed in the last decades, automatic optical-to-synthetic aperture radar (SAR) image matching is still a challenging task due to significant nonlinear intensity differences between such images. This letter addresses this problem by proposing a novel similarity metric for image matching using shape properties. A shape descriptor named dense local self-similarity (DLSS) is first developed based on self-similarities within images. Then a similarity metric (named DLSC) is defined using the normalized cross correlation (NCC) of the DLSS descriptors, followed by a template matching strategy to detect correspondences between images. DLSC is robust against significant nonlinear intensity differences because it captures the shape similarity between images, which is independent of intensity patterns. DLSC has been evaluated with four pairs of optical and SAR images. Experimental results demonstrate its advantage over the state-of-the-art similarity metrics (such as NCC and mutual information), and show the superior matching performance.

Quick and accurate building damage assessment following a disaster is critical to making a preliminary estimate of losses. Remote sensing image analysis based on convolutional neural network and its relatives has shown a growing potential in this task, but faces the challenge of collecting dense pixel-level annotations. In this letter, we propose a novel weakly supervised semantic segmentation (WSSS) method based on image-level labels for pixel-wise damaged building extraction from post-earthquake high resolution remote sensing (HRRS) images. The proposed method aims to improve the quality of the class activation map (CAM) to boost model performance. To be specific, a multi-scale dependence (MSD) module and a spatial correlation refinement (SCR) module are designed by considering the special characteristics of damaged building, and are integrated into an encoder-decoder network. The former is used for complete and dense localization of damaged buildings in CAM, and the latter contributes to noise suppression. Extensive experimental evaluations over three datasets are conducted to confirm the effectives of the proposed approach. Both generated CAMs and extracted damaged buildings results of our methods are better than that of current state-of-the-art methods.

A novel model is presented to address the problem of semantic clustering of geo-objects in very high resolution panchromatic satellite images. The proposed model combines a probabilistic topic model with a multiscale image representation into an automatic framework by embedding both document and scale selections. The probabilistic topic model is used to characterize the statistical distributions of both intraclass appearance and inter-class coherence of geo-objects within documents, i.e., squared sub-images. Because the bag-of-words assumption involved in the probabilistic topic models does not consider the spatial coherence between topic labels, the multiscale image representation is designed to provide a self-adaptive spatial regularization for various geo-object categories. By introducing scale and document selections, the automatic framework integrates the probabilistic topic model and the multiscale image representation to ensure that words on a site should be allocated the same topic label no matter what documents they reside in. Consequently, unlike the traditional method of applying topic models for analyzing satellite images, the process of explicitly generating a set of documents before modeling and then combining multiple labels for a word on a given site is unnecessary. Gibbs sampling is adopted for parameter estimation and image clustering. Extensive experimental evaluations are designed to first analyze the effect of parameters in the proposed model and then compare the results of our model with those of some state-of-the-art methods for three different types of images. The results indicate that the proposed algorithm consistently outperforms these exiting state-of-the-art methods in all of the experiments.

Weakly supervised semantic segmentation (WSSS) methods based on image-level labels can relieve the tedious pixel-level annotation burden, and these methods are mainly based on class activation maps (CAMs). However, it is challenging to generate high-quality CAMs for high-resolution remotely sensed imagery (HRSI). In this article, we propose a WSSS method for building extraction from HRSI using image-level labels. The proposed method, termed as the MSG-SR-Net, integrates two novel modules, i.e., multiscale generation (MSG) and superpixel refinement (SR), to obtain high-quality CAMs so as to provide reliable pixel-level training samples for subsequent semantic segmentation steps. The MSG module is proposed to use global semantic information to guide the learning of multiple features across different levels, and then, respectively, to utilize multilevel features for generating multiscale CAMs. This component can effectively suppress the interference of the class-irrelevant noise and strengthen the use of profitable information in multilevel features. The SR module is designed to take advantage of superpixels to improve multiscale CAMs in target integrity and details preserving. Extensive experiments on two public building datasets demonstrated that the proposed modules made the MSG-SR-Net obtain more integral and accurate CAMs for building extraction. Moreover, experimental results also showed the proposed method achieved excellent performance with over 67% in F1-score, and outperformed other weakly supervised methods in effectiveness and generalization ability.