The existing multimodal image classification methods often suffer from several key limitations: difficulty in effectively balancing local detail and global topological relationships in hyperspectral image (HSI) feature extraction; insufficient multi-scale characterization of terrain features from light detection and ranging (LiDAR) elevation data; and neglect of deep inter-modal interactions in traditional fusion methods, often accompanied by high computational complexity. To address these issues, this paper proposes a comprehensive deep learning framework combining convolutional neural network (CNN), a graph convolutional network (GCN), and wavelet transform for the joint classification of HSI and LiDAR data, including several novel components: a Spectral Graph Mixer Block (SGMB), where a CNN branch captures fine-grained spectral–spatial features by multi-scale convolutions, while a parallel GCN branch models long-range contextual features through an enhanced gated graph network. This dual-path design enables simultaneous extraction of local detail and global topological features from HSI data; a Spatial Coordinate Block (SCB) to enhance spatial awareness and improve the perception of object contours and distribution patterns; a Multi-Scale Elevation Feature Extraction Block (MSFE) for capturing terrain representations across varying scales; and a Bidirectional Frequency Attention Encoder (BiFAE) to enable efficient and deep interaction between multimodal features. These modules are intricately designed to work in concert, forming a cohesive end-to-end framework, which not only achieves a more effective balance between local details and global contexts but also enables deep yet computationally efficient interaction across features, significantly strengthening the discriminability and robustness of the learned representation. To evaluate the proposed method, we conducted experiments on three multimodal remote sensing datasets: Houston2013, Augsburg, and Trento. Quantitative results demonstrate that our framework outperforms state-of-the-art methods, achieving OA values of 98.93%, 88.05%, and 99.59% on the respective datasets.
23 September 2025 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Hyperspectral images have become indispensable for advanced material characterization and environmental monitoring, yet conventional analytical frameworks struggle with the evolving nature of spectral signatures in open-world scenarios. Open-set classification addresses this fundamental limitation by enabling recognition of both known and novel spectral categories during inference. Key technical barriers include rotational invariance in multi-angle acquisitions, multi-scale feature compatibility across spectral resolutions, frequency-domain discriminative decay, and interference from morphologically similar compounds. To overcome these challenges, we propose a Frequency-Domain Multi-Branch Adversarial Routing Open-Set Network (FDMAR-OSNet) integrating four core innovations: Fractional Fourier Transform layers for rotation-equivariant spectral localization, Multi-Branch Dynamic Gate Routing (MBDGR) for uncertainty quantified hierarchical feature fusion, Dual-Frequency Enhancement (DFE) modules separating diagnostic spectral components through learned frequency gates, and a Multi-Scale Adaptive Dynamic Adversarial Spectral (MS-ADAS) mechanism enabling joint spectral-spatial attention refinement. The mathematical co-design of adaptive spectral operators and uncertainty-aware architectures establishes new theoretical foundations for robust open-set analysis in dynamic spectral environments.
31 October 2025 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Remote sensing multimodal data integrates information from various sensors, providing detailed characterization and reliable monitoring capabilities for Earth observation tasks. This data is crucial in environmental surveillance, disaster assessment, and related domains. However, current state-of-the-art methods predominantly rely on category names within prompts. These methods demonstrate reduced effectiveness when processing ambiguous or poorly defined category designations, thus compromising classification accuracy. To overcome these limitations, we propose the Composite Text Prompt Multimodal Synergy Network (CTPMSN). This architecture integrates a text encoder with a remote sensing visual joint encoder and implements dual-level composite text supervision at both categorical and content strata. Our approach enhances inter-class discriminability while effectively capturing intra-class variations. Additionally, we introduce the Text-Visual Prompt Alignment Module (TVPAM) to optimize the association between text prompts and visual features, thereby strengthening their correspondence. Extensive experiments on three benchmark datasets demonstrate the effectiveness of the proposed method, achieving overall accuracies of 93.99 % on MUUFL Gulfport, 93.74 % on Houston 2013, and 91.56 % on Augsburg.
20 April 2026 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Hyperspectral image classification in open environments remains challenging because models must accurately recognize known categories while reliably identifying samples from previously unseen classes. To address this problem, this paper proposes a new learning framework named Meta Guided Intrinsic Prototype Network (MGIPNet) that improves robustness and adaptability in open-set hyperspectral classification. The proposed method introduces a meta-guided network that learns to adjust its feature extraction process across tasks, enabling better generalization under varying spectral conditions. To strengthen representation stability, intrinsic reference patterns are learned from known data and used to guide feature reconstruction, encouraging compact and consistent representations for known classes while amplifying deviations caused by unknown samples. In addition, a token based aggregation strategy is employed to compress high dimensional spatial information into a compact and informative representation, reducing redundancy while preserving essential spectral-spatial characteristics. These components are jointly optimized through complementary objectives to balance accurate classification and reliable unknown detection. Experiments conducted on multiple benchmark hyperspectral datasets demonstrate that the proposed approach consistently outperforms existing methods in terms of classification accuracy, stability across classes, and detection of unknown samples. Experimental results on the Pavia University dataset show that the proposed framework reaches an overall accuracy of 97.17% and a novel-class detection accuracy of 85.04%, confirming that the joint use of adaptive optimization, prototype-guided representation learning, and compact feature abstraction is effective for open-set hyperspectral image classification.
1 January 2026 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
With the increasing availability of satellite imagery and the shortening revisit intervals, efficiently processing satellite hyperspectral images has become a critical task. However, in practice, a large portion of satellite hyperspectral data remains unlabeled, making it difficult to achieve satisfactory classification performance using satellite data alone. Meanwhile, UAV-based platforms offer acquisition flexibility, which facilitates the collection of rich and detailed information. To address these challenges, this paper proposes a method called Sharpness-Aware Minimization with Local-to-Global Feature Enhancement (SAMLFE), which uses UAV hyperspectral images for training to enhance the fine-grained classification performance of satellite hyperspectral images in large scenes. Specifically, a spectral dimension mapping model is first employed to unify UAV and satellite images into a common spectral dimension, thereby mitigating the impact of inconsistent feature representations. Next, a local-to-global feature extraction network is constructed to capture both local details and global semantics. Few-shot learning is applied to extract discriminative features from both the source and target domains within the shared feature space, thereby enhancing the model’s ability to utilize limited labeled data efficiently. Furthermore, a conditional adversarial domain adaptation strategy is adopted to align the feature distributions of the source and target domains, thereby alleviating spectral shift. Meanwhile, the integration of an improved Sharpness-Aware Minimization (ISAM) enhances the model’s robustness across domains. Finally, the K-Nearest Neighbor algorithm is employed to perform accurate classification. Experimental results on multiple datasets demonstrate that the proposed method achieves superior generalization and classification performance in cross-domain hyperspectral image classification. It also outperforms existing methods in terms of feature distribution alignment, robustness of feature extraction, and adaptability to small-sample scenarios.