Semantic segmentation for unmanned aerial vehicle (UAV) remote sensing images has become one of the research focuses in the field of remote sensing at present, which could accurately analyze the ground objects and their relationships. However, conventional semantic segmentation methods based on deep learning require large-scale models that are not suitable for resource-constrained UAV remote sensing tasks. Therefore, it is important to construct a light-weight semantic segmentation method for UAV remote sensing images. With this motivation, we propose a light-weight neural network model with fewer parameters to solve the problem of semantic segmentation of UAV remote sensing images. The network adopts an encoder-decoder architecture. In the encoder, we build a light-weight convolutional neural network model with fewer channels of each layer to reduce the number of model parameters. Then, feature maps of different scales from the encoder are concatenated together after resizing to carry out the multiscale fusion. Moreover, we employ two attention modules to capture the global semantic information from the context and the correlation among channels in UAV remote sensing images. In the decoder part, the model obtains predictions of each pixel through the softmax function. We conducted experiments on the ISPRS Vaihingen dataset, UAVid dataset, and UDD6 dataset to verify the effectiveness of the light-weight network. Our method obtains quality semantic segmentation results evaluated on UAV remote sensing datasets with only 9 M parameters the model owns, which is competitive among popular methods with the same level of parameters.

Currently, an increasing number of convolutional neural networks (CNNs) focus specifically on capturing contextual features (con. feat) to improve performance in semantic segmentation tasks. However, high-level con. feat are biased towards encoding features of large objects, disregard spatial details, and have a limited capacity to discriminate between easily confused classes (e.g., trees and grasses). As a result, we incorporate low-level features (low. feat) and class-specific discriminative features (dis. feat) to boost model performance further, with low. feat helping the model in recovering spatial information and dis. feat effectively reducing class confusion during segmentation. To this end, we propose a novel deep multi-feature learning framework for the semantic segmentation of VHR RSIs, dubbed MFNet. The proposed MFNet adopts a multi-feature learning mechanism to learn more complete features, including con. feat, low. feat, and dis. feat. More specifically, aside from a widely used context aggregation module for capturing con. feat, we additionally append two branches for learning low. feat and dis. feat. One focuses on learning low. feat at a shallow layer in the backbone network through local contrast processing, while the other groups con. feat and then optimizes each class individually to generate dis. feat with better inter-class discriminative capability. Extensive quantitative and qualitative evaluations demonstrate that the proposed MFNet outperforms most state-of-the-art models on the ISPRS Vaihingen and Potsdam datasets. In particular, thanks to the mechanism of multi-feature learning, our model achieves an overall accuracy score of 91.91% on the Potsdam test set with VGG16 as a backbone, performing favorably against advanced models with ResNet101.

Many practical applications of high-resolution remote sensing images (HRRSIs) are based on semantic segmentation. However, due to the complex ground object information contained in remote sensing images, it is difficult to make precise semantic segmentation of HRRSIs. In this letter, we proposed a hierarchical context aggregation network (HCANet) for the semantic segmentation of HRRSIs. The HCANet has an encoder-decoder structure which is similar to UNet. In the HCANet, we designed two Compact Atrous Spatial Pyramid Pooling (CASPP and CASPP+) modules. The CASPP modules replace the copy and crop operation in UNet to extract the multiscale context information of the multisemantic features of ResNet. The CASPP+ module is embedded in the middle layer of HCANet’s decoder to provide a strong aggregation path of contextual information. In the decoder of HCANet, the multiscale context information obtained by CASPP modules is hierarchically merged layer by layer for the semantic segmentation of HRRSIs. We compared our method with several of the most advanced methods on the ISPRS Vaihingen and Potsdam data sets. The final results demonstrate that our method can achieve outstanding performance.

Currently, the most advanced high-resolution remote sensing image (HRRSI) semantic labeling methods rely on deep neural networks. However, HRRSIs naturally have a serious class imbalance problem, which is not yet well solved by the current method. The cross-entropy loss is often used to guide the training of semantic labeling neural networks for HRRSIs, but it is essentially dominated by the major classes in the image, resulting in poor predictions for the minority class. Based on the prediction results, focal loss (FL) effectively suppresses the negative impact of class imbalance in dense object detection by redistributing the loss of each sample. In this article, we thoroughly analyze the inadequacy of FL for semantic labeling, which inevitably introduces confusing-classified examples that are more difficult to classify while suppressing the loss of well-classified examples. Therefore, following the core idea of FL, we redefine the hard examples in semantic labeling of HRRSIs and propose the prediction confusion map to measure the classification difficulty. Based on this, we further propose the calibrated focal loss (CFL) for the semantic labeling of HRRSIs. Finally, we conduct complete experiments on the International Society for Photogrammetry and Remote Sensing Vaihingen and Potsdam datasets to analyze the semantic labeling performance, model uncertainty, and confidence calibration of different loss functions. Experimental results show that CFL can achieve outstanding results compared with other commonly used loss functions without increasing model parameters and training iterations, demonstrating the effectiveness of our method. In the end, combined with our previously proposed HCANet, we further verify the effectiveness of CFL on state-of-the-art network structures.

Semantic segmentation for high-resolution remote-sensing (HRRS) images is one of the most challenging tasks in remote-sensing images understanding. Capturing long-range dependencies in feature representations is crucial for semantic segmentation. Recent graph-based global reasoning networks ( GloRe ) focus on modeling the global contextual relationship between latent nodes based on fully connected graph in interaction space. However, such a dense operation is susceptible to redundant features. Most importantly, it treats each node equally, ignoring the contextual relationship between nodes in graphs. In this work, we propose to explore more effective contextual representations in semantic segmentation by introducing dynamic graph contextual reasoning module over GloRe , dubbed DGCR. It incorporates local semantic information that represents the relationships between nodes to perform long-range contextual reasoning. More specifically, to provide effectively and flexible reasoning in graph-based reasoning approaches, we construct $k$ -nearest neighbor (KNN) graphs rather than fully connected graphs using only the $k$ closest nodes depends on pairwise semantic distance. Extensive experiments on the International Society for Photogrammetry and Remote Sensing (ISPRS) Vaihingen and Potsdam datasets demonstrate the effectiveness and superiority of our proposed DGCR module over other state-of-the-art methods.

Semantic labeling of high-resolution remote sensing images (HRRSIs) has always been an important research field in remote sensing image analysis. However, remote sensing images contain substantial low- and high-level features, which makes them quite difficult to be recognized. In this letter, we proposed a multilevel feature fusion and attention network (MFANet) to adaptively capture and fuse multilevel features in a more effective and efficient manner. Specifically, the backbone of our network is divided into two branches—the detail branch and the semantic branch (SB), where the detail branch extracts low-level features and the SB extracts high-level features. The Deep Atrous Spatial Pyramid (DASPP) module is embedded at the end of the SB to capture multiscale features as a supplement to high-level features. It is worth noting that the feature alignment and fusion (FAF) module is used to align and fuse features from different stages to enhance feature representation. Furthermore, the context attention (CA) module is employed to process feature maps from the two branches to establish contextual dependencies in the spatial dimension and channel dimension, which can help the network focus on more meaningful features. The experiments are carried out on the International Society for Photogrammetry and Remote Sensing (ISPRS) Vaihingen and Potsdam datasets, and the results show that our proposed method has achieved better performance than other state-of-art methods.