Semantic change detection (SCD) represents a challenging task in the interpretation of remote sensing images (RSIs), with the goal of identifying change regions and extracting semantic information from bitemporal RSIs simultaneously. The recent integration of deep neural networks leveraging multitask learning has shown promise in enhancing SCD performance. However, there is still a challenge in improving SCD performance, specifically in designing a fine-grained network structure that can handle the two subtasks of change region localization and semantic information recognition in parallel. In this context, a novel multitask Siamese network, termed EGMS-Net, is proposed to boost the performance of SCD, which consists of three core components. First, a coarse-to-fine multitask Siamese network is constructed to obtain semantic information and change information at multiple levels. Second, an adaptive change information enhancement method based on spatial-spectral collaborative attention mechanism is proposed, which can assist the accurate localization of change regions without significantly increasing the model parameters. Third, a change information guidance module is developed to strengthen the interaction between multitask branches and reduce the difficulty of network training. Experiments on three benchmark datasets demonstrate that the proposed EGMS-Net outperforms existing state-of-the-art methods in the SCD community.
26 June 2023 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
In high-resolution remote sensing images, road scale diversity and occlusions caused by shadows, buildings, and vegetation often pose challenges for road extraction. Currently, end-to-end models constructed using deep convolutional neural networks are widely used in road extraction and have significantly improved the accuracy of this task. However, the connectivity and completeness of their results require improvement. This paper proposes a dual task-driven deep convolutional neural network constructed by combining road shape patterns and scale differences. The mainline task is road-surface segmentation, the encoder of which employs residual convolution for feature extraction. The decoder comprises a multi-scale and multi-direction strip convolution module, the output of which is the final extraction result. The splitting task is road centerline extraction, the input features of which come from the coding layer of the road-surface segmentation branches. The intermediate features are incorporated into the decoder of the road-surface segmentation branches, to fully exploit the road centerline and thus improve the road-surface segmentation result connectivity. Implementation of the proposed method on the CHN6-CUG and DeepGlobe datasets reveals superior performance to comparative methods as regards quantitative evaluation metrics; evident advantages for road coverings, road intersections, and low-scale roads; greater model portability; and better small-sample learning capability.
16 April 2024 in IEEE Transactions on Geoscience and Remote Sensing
Multimodal land use/cover classification using optical and synthetic aperture radar (SAR) images has attracted significant attention because the unique radiation and geometric characteristics of these images provide complementary information regarding land properties. However, the significant differences between these modalities create a large semantic gap, posing challenges for effective feature fusion in multimodal learning. Moreover, missing modalities often occur in practical applications due to weather constraints or sensor malfunctions, posing challenges to achieving high performance in cross-modal learning. In this study, we proposed a multimodal online knowledge distillation (MMOKD) framework, designed for land use/cover classification of optical and SAR images using either full or missing modalities. This framework trains one modality-fusion network alongside two modality-specific networks in an end-to-end manner, facilitating both multimodal and cross-modal learning. More specifically, we developed a multimodal feature fusion (MFF) module for integrating heterogeneous features and a single-modal feature generation (SFG) module for encapsulating cross-modal complementary information. In addition, we proposed the joint distillation with multitype fusion knowledge (JD-MFK) method, guiding the modality-specific student networks to comprehensively learn the modality-fusion teacher network. Notably, we adopted an online distillation strategy for real-time feedback and synchronous updates of both modality-fusion and modality-specific networks. Finally, we conducted extensive experiments on two multimodal land use/classification datasets with advanced multimodal fusion, cross-modal distillation, and specific baseline networks for comparison. The results demonstrate the effectiveness of the proposed MMODD, which not only outperforms the other networks in both full- and missing-modality scenarios but also significantly improves model training efficiency.
1 December 2025 in International Journal of Applied Earth Observation and Geoinformation
Owing to significant differences in synthetic aperture radar (SAR) images caused by diverse imaging mechanisms and imaging conditions, inconsistent features and relationship correspondences constitute key problems for traditional image registration algorithms. This study presents a novel SAR image registration method based on the shape information of distinct ground objects, which is obtained via object extraction and morphological operations. We utilize a shape context descriptor to compare the contours of objects and detect invariant control points. The experimental results show that the proposed method can achieve a reliable and stable registration performance for SAR images of different sensors.