Although the vision transformer-based methods (ViTs) exhibit excellent performance than convolutional neural networks (CNNs) for image recognition tasks, their pixel-level semantic segmentation ability is limited due to the lack of explicit utilization of local biases. Recently, a variety of hybrid structures of ViT and CNN have been proposed, but these methods have poor multi-scale fusion ability and cannot accurately segment high-resolution and high-content complex land cover remote sensing images. Therefore, a dual encoder-decoder network named DEDNet is proposed in this work. In the encoding stage, the local and global information of the image is extracted by parallel CNN encoder and Transformer encoder. In the decoding stage, the cross-stage fusion (CF) module is constructed to achieve neighborhood attention guidance to enhance the positioning of small targets, effectively avoiding intra-class inconsistency. At the same time, the multi-head feature extraction (MFE) module is proposed to strengthen the recognition ability of the target boundary and effectively avoid inter-class ambiguity. Before outputting, the fusion spatial pyramid pooling (FSPP) classifier is proposed to merge the outputs of the two decoding strategies. The experiments demonstrate that the proposed model has superior generalization performance and can handle various semantic segmentation tasks of land cover.
14 May 2024 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
With the advancement of remote sensing (RS) image technology, the availability of very high-resolution (VHR) image data has brought new challenges to change detection (CD). Currently, deep learning-based CD methods commonly employ bitemporal interaction networks using convolutional neural networks (CNNs) or transformers. Yet, these models overly emphasize object accuracy, leading to a significant increase in computational costs with limited performance gains. Additionally, the current bitemporal interaction mechanisms are simplistic, failing to adequately account for spatial positions and scale variations of different objects, resulting in an inaccurate modeling of dynamic feature changes between images. To address these issues, a bitemporal attention sharing network (BASNet) is proposed, which tackles the problems effectively by making bitemporal and multiscale attention sharing the primary mode of feature interaction. Specifically, The proposed bitemporal attention sharing module leverages pairs of features preliminarily encoded by a backbone to construct shared global features, directing attention to target changes. Then, through cross-scale attention guidance and weighted fusion, it achieves attention sharing of multiscale features, eliminating the need for over-relying on deep convolutional layers for feature extraction. Experiments on three public datasets demonstrate that, in comparison to several state-of-the-art methods, our model achieves superior performance with low computational cost. Our code is available at https://github.com/JasonWZC/BASNet .
Change detection is widely used in the field of building monitoring. In recent years, the progress of remote sensing image technology has provided high-resolution data. However, unlike other tasks, change detection focuses on the difference between dual-input images, so the interaction between bi-temporal features is crucial. However, the existing methods have not fully tapped the potential of multi-scale bi-temporal features to interact layer by layer. Therefore, this paper proposes a multi-scale feature interaction network (MFINet). The network realizes the information interaction of multi-temporal images by inserting a bi-temporal feature interaction layer (BFIL) between backbone networks at the same level, guides the attention to focus on the difference region, and suppresses the interference. At the same time, a double temporal feature fusion layer (BFFL) is used at the end of the coding layer to extract subtle difference features. By introducing the transformer decoding layer and improving the recovery effect of the feature size, the ability of the network to accurately capture the details and contour information of the building is further improved. The F1 of our model on the public dataset LEVIR-CD reaches 90.12%, which shows better accuracy and generalization performance than many state-of-the-art change detection models.
Segmenting clouds and their shadows is a critical challenge in remote sensing image processing. The shape, texture, lighting conditions, and background of clouds and their shadows impact the effectiveness of cloud detection. Currently, architectures that maintain high resolution throughout the entire information-extraction process are rapidly emerging. This parallel architecture, combining high and low resolutions, produces detailed high-resolution representations, enhancing segmentation prediction accuracy. This paper continues the parallel architecture of high and low resolution. When handling high- and low-resolution images, this paper employs a hybrid approach combining the Transformer and CNN models. This method facilitates interaction between the two models, enabling the extraction of both semantic and spatial details from the images. To address the challenge of inadequate fusion and significant information loss between high- and low-resolution images, this paper introduces a method based on ASMA (Axial Sharing Mixed Attention). This approach establishes pixel-level dependencies between high-resolution and low-resolution images, aiming to enhance the efficiency of image fusion. In addition, to enhance the effective focus on critical information in remote sensing images, the AGM (Attention Guide Module) is introduced, to integrate attention elements from original features into ASMA, to alleviate the problem of insufficient channel modeling of the self-attention mechanism. Our experimental results on the Cloud and Cloud Shadow dataset, the SPARCS dataset, and the CSWV dataset demonstrate the effectiveness of our method, surpassing the state-of-the-art techniques for cloud and cloud shadow segmentation.