Semantic segmentation of multimodal remote sensing images plays a crucial role in geospatial analysis. However, existing methods often struggle to effectively integrate spatial–spectral and geometric–geographic information, capture multiscale contextual features, and preserve the fine-grained boundaries of geo-objects. To address these challenges, we propose an interaction-guided multimodal fusion network with elevation constraint, referred to as IGECNet. The proposed network comprises four key components. First, a multimodal correlation interaction (MCI) mechanism adaptively fuses features from high-resolution remote sensing images (HRRSIs) and digital surface models (DSMs) using multiscale dilated convolutions and cross-attention mechanisms, thereby enhancing complementary relationships while preserving modality-specific characteristics. Second, a channel reorder enhancement module (CREM) hierarchically decomposes channel-wise features based on entropy and global average pooling values, improving spatial–spectral discriminability. Third, a frequency-guided context decoder (FGCD) refines segmentation outputs by leveraging global semantic context via the Fourier transform and fine boundary details through local feature fusion. In addition, a new loss function, jointly constrained by semantic, boundary, and elevation consistency, enforces elevation consistency from DSMs gradients and boundary alignment. Extensive experiments on the ISPRS Vaihingen, ISPRS Potsdam, and DroneDeploy segmentation benchmarks demonstrate that IGECNet achieves state-of-the-art performance. The source code will be publicly available upon publication.
28 April 2025 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Remote sensing scene classification enables data-driven decisions for various applications, such as environmental monitoring, urban planning, and disaster management. However, deep learning models used for scene classification are highly vulnerable to adversarial samples, resulting in incorrect predictions and posing significant risks. While most current methods focus on improving adversarial robustness, they face a trade-off that compromises accuracy on clean, unperturbed images. To address this challenge, we utilized information theory by incorporating a mutual information (MI) representation (MIR) module, which allows the model to capture high-quality, robust features. Furthermore, a domain adversarial training (AdvT) strategy is applied to promote the learning of domain-invariant features, reducing the effect of distribution differences between clean images and adversarial samples. We propose a novel algorithm that accurately differentiates between clean and adversarial scenes by introducing the MI and domain adaptation-guided network (MIDANet). Extensive experiments demonstrate the effectiveness of our approach against adversarial attacks, revealing a positive correlation between adversarial perturbations and image information entropy, and a negative correlation with robust accuracy (RAcc).
Deep learning models excel in interpreting the exponentially growing amounts of remote sensing data; however, they are susceptible to deception and spoofing by adversarial samples, posing catastrophic threats. The existing methods to combat adversarial samples have limited performance in robustness and efficiency, particularly in complex remote sensing scenarios. To tackle these challenges, an unsupervised domain adaptation algorithm is proposed for the accurate identification of clean images and adversarial samples by exploring a robust generative adversarial classification network that can harmonize the features between clean images and adversarial samples to minimize distribution discrepancies. Furthermore, linear polynomial loss as a replacement for cross-entropy loss is integrated to guide robust representation learning. Additionally, we leverage the fast gradient sign method (FGSM) and projected gradient descent (PGD) algorithms to generate adversarial samples with varying perturbation amplitudes to assess model robustness. A series of experiments was performed on the RSSCN7 dataset and SIRI-WHU dataset. Our experimental results illustrate that the proposed algorithm performs exceptionally well in classifying clean images while demonstrating robustness against adversarial perturbations.
18 June 2025 in International Journal of Computers and Applications