In the context of fast progress in deep learning, convolutional neural networks (CNNs) have been extensively applied to the semantic segmentation of remote sensing images and have achieved significant progress. However, certain limitations exist in capturing global contextual information due to the characteristics of convolutional local properties. Recently, Transformer has become a focus of research in computer vision and has shown great potential in extracting global contextual information, further promoting the development of semantic segmentation tasks. In this paper, we use ResNet50 as an encoder, embed the hybrid attention mechanism into Transformer, and propose a Transformer-based decoder. The Channel-Spatial Transformer Block (CSTB) further aggregates features by integrating the local feature maps extracted by the encoder with their associated global dependencies. At the same time, an adaptive approach is employed to reweight the interdependent channel maps to enhance the feature fusion. The Global Cross-Fusion Module (GCFM) combines the extracted complementary features to obtain more comprehensive semantic information. Extensive comparative experiments were conducted on the ISPRS Potsdam and Vaihingen datasets, where mIoU reached 78.06% and 76.37%, respectively. The outcomes of multiple ablation experiments also validate the effectiveness of the proposed method.

Adding an attention module to the deep convolution semantic segmentation network has significantly enhanced the network performance. However, the existing channel attention module focusing on the channel dimension neglects the spatial relationship, causing location noise to transmit to the decoder. In addition, the spatial attention module exemplified by self-attention has a high training cost and challenges in execution efficiency, making it unsuitable to handle large-scale remote sensing data. We propose an efficient vector pooling attention (VPA) module for building the channel and spatial location relationship. The module can locate spatial information better by performing a unique vector average pooling in the vertical and horizontal dimensions of the feature maps. Furthermore, it can also learn the weights directly by using the adaptive local cross-channel interaction. Multiple weight learning ablation studies and comparison experiments with the classical attention modules were conducted by connecting the VPA module to a modified DeepLabV3 network using ResNet50 as the encoder. The results show that the mIoU of our network with the addition of an adaptive local cross-channel interaction VPA module increases by 3% compared to the standard network on the MO-CSSSD. The VPA-based semantic segmentation network can significantly improve precision efficiency compared with other conventional attention networks. Furthermore, the results on the WHU Building dataset present an improvement in IoU and F1-score by 1.69% and 0.97%, respectively. Our network raises the mIoU by 1.24% on the ISPRS Vaihingen dataset. The VPA module can also significantly improve the network’s performance on small target segmentation.

Convolutional neural networks have attracted much attention for their use in the semantic segmentation of remote sensing imagery. The effectiveness of semantic segmentation of remote sensing images is significantly influenced by contextual information extraction. The traditional convolutional neural network is constrained by the size of the convolution kernel and mainly concentrates on local contextual information. We suggest a new lightweight global context semantic segmentation network, LightFGCNet, to fully utilize the global context data and to further reduce the model parameters. It uses an encoder–decoder architecture and gradually combines feature information from adjacent encoder blocks during the decoding upsampling stage, allowing the network to better extract global context information. Considering that the frequent merging of feature information produces a significant quantity of redundant noise, we build a unique and lightweight parallel channel spatial attention module (PCSAM) for a few critical contextual features. Additionally, we design a multi-scale fusion module (MSFM) to acquire multi-scale feature target information. We conduct comprehensive experiments on the two well-known datasets ISPRS Vaihingen and WHU Building. The findings demonstrate that our suggested strategy can efficiently decrease the number of parameters. Separately, the number of parameters and FLOPs are 3.12 M and 23.5 G, respectively, and the mIoU and IoU of our model on the two datasets are 70.45% and 89.87%, respectively, which is significantly better than what the conventional convolutional neural networks for semantic segmentation can deliver.

Contextual information can effectively aid deep learning models in extracting inter-class and intra-class difference features in remote sensing images. Our research presents a novel approach called the adaptive attention network (AANet) for semantic segmentation in high-resolution remote sensing images. The proposed AANet aims to enhance the segmentation performance while minimizing the network's computational and parametric aspects. Furthermore, the AANet is designed to facilitate real-time segmentation. The AANet involves the construction of three distinct modules, namely the Multiscale Channel Attention Module (MCAM), the Multidimensional Spatial Attention Module (MSAM), and the Contextual Information Adaptive Fusion Module (CIAFM). MCAM enhances a multi-scale approach to effectively capture contextual information from neighboring channels and category information. MSAM is designed to extract and combine detailed information from each dimension of the spatial domain. CIAFM focuses on the complementary nature of channel and spatial context information and the correlation between pixels and categories. The methodology employed in this study involved conducting experiments on the ISPRS Vaihingen, ISPRS Potsdam, and MO-CSSSD datasets alongside a comparative analysis with conventional semantic segmentation models. The results of the study indicate that our approach demonstrates exceptional performance on the ISPRS Vaihingen dataset, ISPRS Potsdam dataset, and MO-CSSSD dataset, achieving mIoU scores of 83.17%, 85.67%, and 89.68%, respectively.