24 March 2021 in IEEE Transactions on Geoscience and Remote Sensing
Semantic segmentation in very-high-resolution (VHR) aerial images is one of the most challenging tasks in remote sensing image understanding. Most of the current approaches are based on deep convolutional neural networks (DCNNs). However, standard convolution with local receptive fields fails in modeling global dependencies. Prior research works have indicated that attention-based methods can capture long-range dependencies and further reconstruct the feature maps for better representation. Nevertheless, limited by the mere perspective of spatial and channel attention and huge computation complexity of self-attention (SA) mechanism, it is unlikely to model the effective semantic interdependencies between each pixel pair of remote sensing data with complex spectra. In this work, we propose a novel attention-based framework named hybrid multiple attention network (HMANet) to adaptively capture global correlations from the perspective of space, channel, and category in a more effective and efficient manner. Concretely, a class augmented attention (CAA) module embedded with a class channel attention (CCA) module can be used to compute category-based correlation and recalibrate the class-level information. In addition, we introduce a simple yet effective region shuffle attention (RSA) module to reduce feature redundant and improve the efficiency of SA mechanism via regionwise representations. Extensive experimental results on the ISPRS Vaihingen, Potsdam benchmark, and iSAID data set demonstrate the effectiveness and efficiency of our HMANet over other state-of-the-art methods.
19 October 2021 in IEEE Transactions on Geoscience and Remote Sensing
Semantic segmentation in aerial imagery is still an important, yet challenging task due to the complex characteristics of remote-sensing data. The critical issues consist of: 1) extreme foreground–background imbalance; 2) large intra-class variance; and 3) arbitrary-oriented, dense, and small objects. The above challenges make it unlikely to model the effective global interdependencies of semantic heterogeneous regions. Besides, general semantic segmentation methods suffer from feature ambiguity due to the joint feature learning paradigm, leading to inferior detail information. In this article, we propose an improved semantic segmentation framework to tackle these problems via graph reasoning (GR) and disentangled learning. On the one hand, a simple, yet effective GR unit is introduced to implement coordinate-interaction space mapping and perform relation reasoning over the graph. It can be deployed on the feature pyramid network (FPN) to exploit cross-stage multi-scale information. On the other hand, we propose a so- called disentangled learning paradigm to explicitly model the foreground and boundary objects, instantiated as foreground prior estimation (FPE) and boundary alignment (BA). The indication of the intermediate feature can be effectively emphasized to enhance the discriminative abilities of the network. Extensive experiments over iSAID, ISPRS Vaihingen, and the general Cityscapes datasets demonstrate the effectiveness and efficiency of the proposed framework over other state-of-the-art semantic segmentation methods.