Existing optical remote sensing image change detection (CD) methods aim to learn an appropriate discriminate decision by analyzing the feature information of bitemporal images obtained at the same place. However, the complex scenes in high-resolution (HR) remote images cause unsatisfied results, especially for some irregular and occluded objects. Although recent self-attention-driven change detection models with CNN achieve promising effects, the computational and consumed parameters costs emerge as an impassable gap for HR images. In this paper, we utilize a transformer structure replacing self-attention to learn stronger feature representations per image. In addition, concurrent vision transformer models only consider tokenizing single-dimensional image tokens, thus failing to build multi-scale long-range interactions among features. Here, we propose a hybrid multi-scale transformer module for HR remote images change detection, which fully models representation attentions at hybrid scales of each image via a fine-grained self-attention mechanism. The key idea of the hybrid transformer structure is to establish heterogeneous semantic tokens containing multiple receptive fields, thus simultaneously preserving large object and fine-grained features. For building relationships between features without embedding with token sequences from the Siamese tokenizer, we also introduced a hybrid difference transformer decoder (HDTD) layer to further strengthen multi-scale global dependencies of high-level features. Compared to capturing single-stream tokens, our HDTD layer directly focuses representing differential features without increasing exponential computational cost. Finally, we propose a cascade feature decoder (CFD) for aggregating different-dimensional upsampling features by establishing difference skip-connections. To evaluate the effectiveness of the proposed method, experiments on two HR remote sensing CD datasets are conducted. Compared to state-of-the-art methods, our Hybrid-TransCD achieved superior performance on both datasets (i.e., LEVIR-CD, SYSU-CD) with improvements of 0.75% and 1.98%, respectively.
1 January 2021 in International Journal of Computational Intelligence Systems
COVID-19 is an infectious disease caused by virus SARS-CoV-2 virus. Early classification of COVID-19 is essential for disease cure and control. Transcription-polymerase chain reaction (RT-PCR) is used widely for the detection of COVID-19. However, its high cost, time-consuming and low sensitivity will significantly reduce the diagnosis efficiency and increase the difficulty of diagnosis for COVID-19. For X-ray images of COVID-19 patients have high inter-class similarity and low intra-class variability, we specifically designed a multi attention interaction enhancement module (MAIE) and proposed a new convolutional neural network, MAI-Net, based on this module. As a lightweight network, MAI-Net has fewer layers and amount of network parameters than other network models, enabling more efficient detection of COVID-19. To verify the performance of the model, MAI-Net performed a comparison experiment on two open-source datasets. The experimental results show that its overall accuracy and COVID-19 category accuracy are 96.42% and 100%, respectively, and the sensitivity of COVID-19 is 99.02%. Considering the factors such as accuracy rate, the parameters number of network model and the calculation amount, MAI-Net has better practicability. Compared with the existing work, the network structure of MAI-Net is simpler, and the hardware requirements of the equipment are lower, which can be better used in ordinary equipment.
21 September 2021 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Change detection methods for optical remote sensing images play an important role in environmental resource management. Although recent methods based on deep learning demonstrate incredible ability by constructing networks, first, extracting bitemporal features in a separate manner; second, fusing bitemporal images before forwarding them into the single-level network. Both severely neglect the effect of spatial-temporal feature correlation between bitemporal images. In addition, most existing methods represent multiscale feature pairs in a layer-wise manner like ResNet, failing to consider the inner multilevel structure. In this work, we propose a new siamese change detection feature encoder backbone named cross-siamese Res2Net (CSRes2Net), by establishing crossed and hierarchical residual-like connections within one single residual block. The CSRes2Net represents dual features in a fine-grained manner and fully leads to the flow of bitemporal features. In addition, recent learning-based methods designed some spatial-temporal relation modules to capture the pixel-level pairwise relationship and channel dependency based on self-attention mechanism, but they only consider spatial and channel dimension corrections separately with excessive parameters. So we propose a lightweight cross spatial-channel triplet attention module to capture cross-dimensional long-range relationship between triplet combinations: channel with height, channel with width, channel with channel. Finally, we propose a hierarchical-split block for generating multiscale feature representations in a coarse-to-fine fashion. The experiments results on LEVIR-CD and season-varying change detection dataset outperform most state-of-the-art models.
16 August 2022 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Remote sensing image superresolution has made significant progress in recent years, aiming to restore natural and realistic high-resolution images from low-resolution images. However, most image superresolution remote sensing methods are improved only by deepening their network and expanding the network size, consuming substantial computing resources and imposing a bottleneck in development. Here, we propose an end-to-end image superresolution network called texture transfer transformer for remote sensing image superresolution (T $^{3}$ SR). For the first time, T $^{3}$ SR introduces image texture transfer into remote sensing, which achieves the most advanced results. Specifically, T $^{3}$ SR divides image superresolution into two stages: texture transfer and feature fusion. First, to solve the problems of missing textures, artifacts, and blurring in a single image superresolution approach, we design a texture transfer module to serve the shallow texture transfer. Second, to further reduce the dependence of the model on the reference image, we propose a U-Transformer-based feature fusion scheme to reduce the dependence on the reference image. Finally, we conduct numerous experiments on standard public datasets to fully evaluate our approach. In addition to verifying the method's superiority based on the reference image paradigm, we also test the performance without the reference image. All results show that our method yields an abundant texture and finish with better visual results. Moreover, the best score is also obtained in the quantitative parameters of PSNR and SSIM. Compared with the best available approach, T $^{3}$ SR has an improved performance by 0.79 dB and 0.33 dB in the datasets of WHU-RS19 and RSSCN7, respectively.
Change detection based on bi-temporal remote sensing images has made significant progress in recent years, aiming to identify the changed and unchanged pixels between a registered pair of images. However, most learning-based change detection methods only utilize fused high-level features from the feature encoder and thus miss the detailed representations that low-level feature pairs contain. Here we propose a multi-level change contextual refinement network (MCCRNet) to strengthen the multi-level change representations of feature pairs. To effectively capture the dependencies of feature pairs while avoiding fusing them, our atrous spatial pyramid cross attention (ASPCA) module introduces a crossed spatial attention module and a crossed channel attention module to emphasize the position importance and channel importance of each feature while simultaneously keeping the scale of input and output the same. This module can be plugged into any feature extraction layer of a Siamese change detection network. Furthermore, we propose a change contextual representations (CCR) module from the perspective of the relationship between the change pixels and the contextual representation, named change region contextual representations. The CCR module aims to correct changed pixels mistakenly predicted as unchanged by a class attention mechanism. Finally, we introduce an effective sample number adaptively weighted loss to solve the class-imbalanced problem of change detection datasets. On the whole, compared with other attention modules that only use fused features from the highest feature pairs, our method can capture the multi-level spatial, channel, and class context of change discrimination information. The experiments are performed with four public change detection datasets of various image resolutions. Compared to state-of-the-art methods, our MCCRNet achieved superior performance on all datasets (i.e., LEVIR, Season-Varying Change Detection Dataset, Google Data GZ, and DSIFN) with improvements of 0.47%, 0.11%, 2.62%, and 3.99%, respectively.
19 July 2024 in IEEE Geoscience and Remote Sensing Letters
Change detection (CD) is a crucial task in earth remote sensing, and it plays a significant role in analyzing changes in a region. However, bi-temporal images acquisition may cause domain shift problems due to the seasonal difference. Most cross-seasonal change detection methods currently in use make it difficult to capture the impact of domain shift. In contrast, domain adaptation (DA) methods struggle with fuzzy semantic correspondence and suboptimal use of generated data, which leads to incomplete target images. To address these problems, we propose CSDACD, an end-to-end domain-adaptive change detection network that integrates domain-adaptive images into the CD network. Additionally, we introduce the Semantic Change Alignment (SCA) module to include the registration and semantic change information of bi-temporal images and design three data fusion modules to enhance the adaptability of DA and CD networks. Our experiments show that CSDACD outperforms the state-of-the-art (SOTA) methods in cross-seasonal change detection tasks, achieving an F1-score of 98.73% and IoU of 97.49% in the CDD dataset. Source code and models are available at https://github.com/24kironhead/CSDACD.