Building change detection (BCD) for very-high-spatial-resolution (VHR) remote sensing images is very important and challenging in the field of remote sensing, as the building is one of the most significant and valuable man-made ground targets. This article proposes a local–global pyramid network (LGPNet) that combines a local feature pyramid module (LFPM) and a global spatial pyramid module (GSPM) for various building feature extraction. The LFPM is constructed using the convolutional kernel with three different pyramid scales, and then, the local pyramid features are obtained by adding features of each scale. In the GSPM, the global spatial pyramid features are extracted by adaptive average pooling to acquire global contextual information from different fields of view on deep features. The LFPM and the GSPM work in a parallel and complementary manner to capture discriminative features of various buildings. In addition to the LFPM and the GSPM, the proposed LGPNet also employs two general attention mechanisms, i.e., the position attention module and the channel attention module, which can select and emphasize adaptively some building features with high semantic responses. Besides, in order to mitigate the influence of other ground targets to a certain extent, a cross-task transfer learning strategy is introduced to make the LGPNet focus on the building, which significantly improves the performance of our method. Extensive experiments on two public available BCD datasets show that the proposed LGPNet can achieve significant improvement compared with eight other state-of-the-art methods. The source code and the pretrained model will be released at https://github.com/TongfeiLiu/LGPNet .

Feature extraction (FE) is a crucial research area in hyperspectral image (HSI) processing. Recently, due to the powerful ability of deep learning (DL) to extract spatial and spectral features, DL-based FE methods have shown great potentials for HSI processing. However, most of the DL-based FE methods are supervised, and the training of them suffers from the absence of labeled samples in HSIs severely. The training issue of supervised DL-based FE methods limits their application on HSI processing. To address this issue, in this paper, a novel modified generative adversarial network (GAN) is proposed to train a DL-based feature extractor without supervision. The designed GAN consists of two components, which are a generator and a discriminator. The generator can focus on the learning of real probability distributions of data sets and the discriminator can extract spatial-spectral features with superior invariance effectively. In order to learn upsampling and downsampling strategies adaptively during FE, the proposed generator and discriminator are designed based on a fully deconvolutional subnetwork and a fully convolutional subnetwork, respectively. Moreover, a novel min-max cost function is designed for training the proposed GAN in an end-to-end fashion without supervision, by utilizing the zero-sum game relationship between the generator and discriminator. Besides, the proposed modified GAN replaces the original Jensen-Shannon divergence with the Wasserstein distance, aiming to mitigate the unstability and difficulty of the training of GAN frameworks. Experimental results on three real data sets validate the effectiveness of the proposed method.

Three-dimensional point cloud registration is an important field in computer vision. Recently, due to the increasingly complex scenes and incomplete observations, many partial-overlap registration methods based on overlap estimation have been proposed. These methods heavily rely on the extracted overlapping regions with their performances greatly degraded when the overlapping region extraction underperforms. To solve this problem, we propose a partial-to-partial registration network (RORNet) to find reliable overlapping representations from the partially overlapping point clouds and use these representations for registration. The idea is to select a small number of key points called reliable overlapping representations from the estimated overlapping points, reducing the side effect of overlap estimation errors on registration. Although it may filter out some inliers, the inclusion of outliers has a much bigger influence than the omission of inliers on the registration task. The RORNet is composed of overlapping points’ estimation module and representations’ generation module. Different from the previous methods of direct registration after extraction of overlapping areas, RORNet adds the step of extracting reliable representations before registration, where the proposed similarity matrix downsampling method is used to filter out the points with low similarity and retain reliable representations, and thus reduce the side effects of overlap estimation errors on the registration. Besides, compared with previous similarity-based and score-based overlap estimation methods, we use the dual-branch structure to combine the benefits of both, which is less sensitive to noise. We perform overlap estimation experiments and registration experiments on the ModelNet40 dataset, outdoor large scene dataset KITTI, and natural data Stanford Bunny dataset. The experimental results demonstrate that our method is superior to other partial registration methods. Our code is available at https://github.com/superYuezhang/RORNet .

Hyperspectral images (HSIs) contain rich spectral signatures that reveal more image details and, thus, enable the detection of less noticeable changes on the ground. However, HSI-based change detection (CD) is susceptible to a large amount of irrelevant or noisy spectral and spatial information due to massive spectral bands. To address these issues, we propose a novel spectral and spatial attention network (S ² AN) for HSI-based CD, which is capable to suppress CD-irrelevant spectral and spatial information via adaptive spectral and spatial attention mechanisms. S ² AN takes as input the image patch from the difference map between two HSIs and outputs the status of change for the patch. Specifically, S ² AN is composed of several repeated attention blocks, each of which contains the spectral attention (SpeA) module for directly calculating the attention score for each input channel, the Gaussian spatial attention (GSpaA) module that first constructs an adaptive Gaussian distribution and then samples it to derive the attention scores for each spatial position, and the convolutional feature extraction (CFE) module for extracting features from the attention-weighted input. It is worth mentioning that, in addition to the advantage of the attention, GSpaA also reduces the sensitivity of patch size for patch-based methods. To effectively train S ² AN when facing insufficient labeled data, a semisupervised strategy that combines supervised and unsupervised methods to augment labeled training data is proposed. Experiments on several HSI datasets in comparison to existing methods show the superiority of S ² AN.

Extracting 3D information from a single optical image is very attractive. Recently emerging self-supervised methods can learn depth representations without using ground truth depth maps as training data by transforming the depth prediction task into an image synthesis task. However, existing methods rely on a differentiable bilinear sampler for image synthesis, which results in each pixel in a synthetic image being derived from only four pixels in the source image and causes each pixel in the depth map to perceive only a few pixels in the source image. In addition, when calculating the photometric error between a synthetic image and its corresponding target image, existing methods only consider the photometric error within a small neighborhood of each single pixel and therefore ignore correlations between larger areas, which causes the model to tend to fall into the local optima for small patches. In order to extend the perceptual area of the depth map over the source image, we propose a novel multi-scale method that downsamples the predicted depth map and performs image synthesis at different resolutions, which enables each pixel in the depth map to perceive more pixels in the source image and improves the performance of the model. As for the locality of photometric error, we propose a structural similarity (SSIM) pyramid loss to allow the model to sense the difference between images in multiple areas of different sizes. Experimental results show that our method achieves superior performance on both outdoor and indoor benchmarks.

The main challenge of multimodal change detection (MCD) is that multimodal bitemporal images (MBIs) cannot be compared directly to identify changes. To overcome this problem, this paper proposes a novel commonality feature representation learning (CFRL) and constructs a CFRL-based unsupervised MCD framework. The CFRL is composed of a Siamese-based encoder and two decoders. First, the Siamese-based encoder can map original MBIs in the same feature space for extracting the representative features of each modality. Then, the two decoders are used to reconstruct the original MBIs by regressing themselves, respectively. Meanwhile, we swap the decoders to reconstruct the pseudo-MBIs to conduct modality alignment. Subsequently, all reconstructed images are input to the Siamese-based encoder again to map them in a same feature space, by which representative features are obtained. On this basis, latent commonality features between MBIs can be extracted by minimizing the distance between these representative features. These latent commonality features are comparable and can be used to identify changes. Notably, the proposed CFRL can be performed simultaneously in two modalities corresponding to MBIs. Therefore, two change magnitude images (CMIs) can be generated simultaneously by measuring the difference between the commonality features of MBIs. Finally, a simple threshold algorithm or a clustering algorithm can be employed to divide CMIs into binary change maps. Extensive experiments on six publicly available MCD datasets show that the proposed CFRL-based framework can achieve superior performance compared with other state-of-the-art approaches.