Deep learning methods have been extensively applied to hyperspectral (HS) image change detection (CD) tasks and achieved promising performance. However, the beneficial joint spatial–spectral–temporal information provided by the HS images has not been fully used. Since the bi-temporal HS images are highly symmetric, in this letter, we propose a novel cross-temporal interaction symmetric attention network (CSANet), which can effectively extract and integrate the joint spatial–spectral–temporal features of the HS images, at the same time to enhance feature discrimination ability of the changes. Specifically, we propose a novel cross-temporal interaction symmetric attention (CSA) module to interact with the bi-temporal HS information, where self-attention is combined to enhance the feature representation ability of each temporal image, and the cross-temporal attention is utilized to integrate the difference features oriented from each temporal feature embedding. On this basis, we design a Siamese network structure equipped with the CSA to hierarchically extract the change information in a symmetric pattern. Experimental results on three public HS image CD datasets show that the proposed CSANet CD framework achieves a significant improvement when compared with the state-of-the-art (SOTA) methods. The source code of the proposed framework will be released at https://github.com/srxlnnu .
23 June 2022 in IEEE Transactions on Geoscience and Remote Sensing
Convolutional neural networks (CNNs) have been extensively applied to hyperspectral (HS) image classification tasks and achieved promising performance. However, for CNN-based HS image classification methods, it is hard to depict the dependencies among HS image pixels in long-range distanced positions and bands. Moreover, the limited receptive field of the convolutional layers extremely hinders the development of the CNN structure. To tackle these problems, in this article, the novel bottleneck spatial–spectral transformer (BS2T) is proposed to depict the long-range global dependencies of HS image pixels, which can be regarded as a feature extraction module for HS image classification networks. More specifically, inspired by bottleneck transformer in computer vision, for HS image feature extraction, the proposed BS2T is incorporated with a feature contraction module, a multihead spatial–spectral self-attention (MHS2A) module, and a feature expansion module. In this way, convolutional operations are replaced by the MHS2A to capture the long-range dependency of HS pixels regardless of their spatial position and distance. Meanwhile, in the MHS2A module, to highlight the spectral features of HS images, we introduce the spectral information and content spatial positional information to classical multihead self-attention to make the attention more positional aware and spectral aware. On this basis, a dual-branch HS image classification framework based on 3-D CNN and BS2T is defined for jointly extracting the local–global features of HS images. Experimental results on three public HS image classification datasets show that the proposed classification framework achieves a significant improvement when compared with the state-of-the-art methods. The source code of the proposed framework can be downloaded from https://github.com/srxlnnu/BS2T .
17 April 2023 in Computers in Biology and Medicine
With the development of deep learning, convolutional neural networks (CNNs) have been successfully applied in the field of change detection in heterogeneous remote sensing (RS) images and achieved remarkable results. However, most of the existing methods of heterogeneous RS image change detection only extract deep features to realize the whole image transformation and ignore the description of the topological structure composed of the image texture, edge, and direction information. The occurrence of change often means that the topological structure of the ground object has changed. As a result, these algorithms severely limit the performance of change detection. To solve these problems, this paper proposes a new topology-coupling-based heterogeneous RS image change detection network (TSCNet). TSCNet transforms the feature space of heterogeneous images using an encoder–decoder structure and introduces wavelet transform, channel, and spatial attention mechanisms. The wavelet transform can obtain the details of each direction of the image and effectively capture the image’s texture features. Unnecessary features are suppressed by allocating more weight to areas of interest via channels and spatial attention mechanisms. As a result of the organic combination of a wavelet, channel attention mechanism, and spatial attention mechanism, the network can focus on the texture information of interest while suppressing the difference of images from different domains. On this basis, a bitemporal heterogeneous RS image change detection method based on the TSCNet framework is proposed. The experimental results on three public heterogeneous RS image change detection datasets demonstrate that the proposed change detection framework achieves significant improvements over the state-of-the-art methods.
Hyperspectral images are valuable for precise land cover change detection in a consistent area over time. Nevertheless, supervised methods for hyperspectral change detection face limitations due to insufficient labeled samples. Additionally, current deep learning-based approaches often neglect critical feature interactions across temporal, spatial, and spectral domains. To address these challenges, we introduce a semi-supervised model called the HyperMatch-based Cross Temporal and Spatial Interaction Transformer (CTSIT) for hyperspectral change detection. The key contributions of this study are as follows: 1) Introduction of the HyperMatch training schedule, which relies on weak-to-strong consistency, to enhance feature extraction using both labeled and unlabeled data. 2) Contribution of the Cross Temporal Bidirectional Attention (CTBA) module to emphasize bidirectional temporal interactions and resemblances. 3) Introduction of the Spatial and Spectral Attention (SSA) module, which includes the Dense Cross Spatial Attention (DCSA) and Spectral Attention (SA) modules, to capture long-range densely spatial interactions and internal spectral similarities. Extensive comparative experiments conducted on three mainstream hyperspectral change detection datasets confirm the effectiveness and superiority of the proposed HyperMatch-based CTSIT in utilizing both labeled and unlabeled samples. For instance, the training ablation demonstrates that this method significantly outperforms most existing state-of-the-art (SOTA) hyperspectral change detection methods, even with a significantly smaller number of samples.