18 September 2023 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
With the improvement of spatial resolution, the conveyed information of remote sensing images has become increasingly intricate. The semantic content of pixels within the same object exhibits considerable variability, while the semantic content of pixels between different objects exhibits significant overlap. However, most existing approaches focus solely on establishing the internal consistency of objects by aggregating global or multi-scale contextual information without adequately considering the orientation and spatially detailed features of the target. Moreover, these methods often overlook the potential of edge information in achieving accurate edge positioning. These defects will adversely affect the accuracy of segmentation. In this paper, we present an Edge Information Guided Network (EIGNet), which leverages edge information to guide the aggregation of rich contextual information for semantic segmentation to improve the segmentation accuracy of high-resolution remote sensing images. Specifically, an Orientation Convolution Module (OCM) is proposed to construct a spatial detail branch for acquiring precise edge information and spatial detail information. To effectively guide the aggregation of spatial detail features and semantic features, we propose a spatial-semantic feature aggregation module (SSFAM). Moreover, to enhance the extraction of long-range dependencies of irregular objects, we propose the Orientation Atrous Convolution Module (OACM), which facilitates the extraction of multi-phase long-range dependencies of objects. The ISPRS Vaihingen and Potsdam datasets are employed to validate the efficacy of the proposed methodology and draw comparisons with various state-of-the-art techniques. The experimental results demonstrate that the proposed method offers distinct advantages.
25 September 2023 in IEEE Geoscience and Remote Sensing Letters
Geospatial object segmentation is a fundamental task in remote sensing image interpretation. Although deep learning has shown great potential for this task, it often suffers from limited receptive fields, insufficient global feature extraction ability, and inaccurate edge positioning, resulting in low accuracy and errors in the results. In this letter, we propose a novel Global Context Dependency Awareness Network (GCDNet) to achieve high-accuracy segmentation results. To overcome the limited receptive fields and promote feature extraction ability, we propose a new dot product attention mechanism to establish long-distance dependencies between different receptive field feature maps. To achieve more accurate object edges, we design an edge-aware optimization module to guide the operation to directly optimize the edge details from the prediction result at the pixel level. Extensive experiments on two well-known public high-resolution remote sensing image datasets have been conducted to verify the performance of the proposed method, and the results show that the proposed method has significant advantages and maintains its robustness in different cases. Code will be available at: https://github.com/Cuiadd/GCDNet.
1 October 2024 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Multiple modal data fusion can provide valuable and diverse information for remote sensing image segmentation. However, existing fusion methods often lead to feature loss during the fusion of various modal data, and the complementarity among multimodal features is insufficient. To address these problems, we propose a multi-modal gated segmentation network for remote sensing images based on frequency decomposition. Complementary information from multimodal features is extracted by establishing a long-distance correlation between the low-frequency components of different modal data. Additionally, high-frequency detailed features of different modal data are preserved by residual connection. The adaptive gated fusion method is then used to control the information flow between the complementary information and each modality feature map, enabling adaptive fusion between multimodal features. These operations can effectively improve the adaptability of the proposed method in various scenarios and data changes. Extensive experiments demonstrate that the proposed method has good effectiveness, robustness, and generalization and achieved state-of-the-art performance in several remote sensing image semantic segmentation tasks. The code will be available at: https://github.com/Cuiadd/ FDGSNet.
4 March 2025 in IEEE Transactions on Geoscience and Remote Sensing
Multimodal data fusion can provide valuable and diverse information for remote sensing image segmentation. However, different modal data have different feature distributions, which causes some conflicts and redundancies in cross-modal feature fusion. In addition, existing multimodal fusion networks usually adopt a two-branch structure with a large number of parameters and high computational cost. To address these problems, we propose a multi-scale cross-modal knowledge transfer network for remote sensing image segmentation. First, we use a cross-modal migration learning method that combines channel discrete loss and spatial discrete loss to facilitate cross-modal migration of geometric and semantic features and reduces redundancy by minimizing the differences in feature distribution. Then, we use the polarized cross-self-attention mechanism to establish long-range correlations between different modal features across spatial and channel dimensions, and only a small number of parameters need to be added to achieve complementary cross-modal feature fusion. Finally, to accurately capture object edges, we propose a multi-scale edge perception module to optimize the edge details in the prediction results at the pixel level. Extensive experiments demonstrate that the proposed method is effective, robust, and generalizable, and achieves state-of-the-art performance in multiple remote sensing image semantic segmentation tasks with only 13.51 million parameters. The code will be available at https://github.com/NUAALISILab/MCKTNet.
30 August 2023 in IEEE Transactions on Geoscience and Remote Sensing
Infrared small target detection is crucial in military applications such as guidance, early warning, and UAV detection. Errors in infrared small target detection are classified as either miss detection (MD) or false alarm (FA). An effective detector should minimize both MD and FA. However, conventional approaches often rely on a single strategy to reduce overall detection errors, which can result in either MD or FA. To address this, we propose a dual-branch gate-aware network (DBGNet) model, that consists of two branches, each learning feature to reduce MD and FA, respectively. Specifically, a multi-scale full convolutional network (MFCN) is first applied to extract different level features to preserve the information of small infrared targets. Additionally, we introduce a multi-scale space and channel gate fusion module (MSCGFM) to ensure the independence of the two branches. Each branch is associated with its own learning objective loss function, enabling them to learn distinct discriminations while being constrained by the same category labels. Moreover, the features from both branches are fused to create a feature representation for each pixel in the image, addressing both MD and FA and minimizing MD while also reducing FA. Finally, the fused features from the two branches are passed through a classification head to generate prediction results. Extensive experimental results demonstrate that DBGNet outperforms other methods on three existing infrared small target datasets.
6 March 2023 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Feature selection is an important way to achieve high precision and efficient classification of hyperspectral remote sensing images. However, most existing FS methods use a fixed scale to extract features and the relationship between spatial and spectral dimensions is ignored. In fact, this correlation is useful for classification. In this paper, a multi-scale feature fusion network (MSFGW) is proposed in which a global weighting mechanism is explored to catch spatial-spectral information at multiple scales. First, the multi-scale feature extraction module composed of group convolution and dilated convolution is utilized to extract the multi-scale features. With the increase of the dilation rate, the module takes the spatial differences at varying scales. Secondly, a 3D weighting mechanism is used to combine the spatial and spectral correlated information for reducing the interference of homologous and heterologous and boosting the feature discrimination ability. Then, multi-scale weighted features are fused to integrate the internal information of all bands at different scales. Finally, the band reconstruction network is used to select representative bands according to their entropy. The experimental results with the state-of-the-art feature selection algorithms on three widely hyperspectral datasets demonstrate that the features selected by MSFGW have obvious advantages in classification with only a few training samples.