In hyperspectral image (HSI) classification, Transformer and CNN are widely used because they complement each other in extracting features. Nevertheless, existing Transformer-based methods still struggle to capture fine-grained local spatial details and positional relationships. Meanwhile, CNN-based attention mechanisms typically process all spectral bands simultaneously to obtain global features, often overlooking locally salient information. To address these challenges, we propose an innovative normalized grouping fused comprehensive relative position transformer network (NG-CRPTN). Specifically, NG-CRPTN comprises two core modules: the comprehensive relative position module and the normalized mutual information band division. The former employs tokens of different dimensions as its foundation, modeling local-to-global spatial structures and relative positional relationships through the local independent relative position transformer and global independent relative position transformer, while incorporating a local complementary enhancement fusion module to deeply integrate high and low-frequency information from spatial features across different dimensions. The latter achieves adaptive band grouping of HSI spectral bands to better focus on locally salient features in the spectral dimension. Extensive experiments conducted on four HSI datasets demonstrate that the proposed NG-CRPTN achieves superior classification accuracy with relatively few training samples compared to existing state-of-the-art methods.
24 October 2024 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Hyperspectral image (HSI) is pivotal in ground object classification, owing to its rich spatial and spectral information. Recently, Convolutional neural networks (CNNs) and Graph Neural Networks (GNNs) have become hotspots in HSI classification. Although various methods have been developed, the problem of detail loss may still exist when extracting complex features within homogenous regions. To solve this issue, we proposed a double-hop graph attention multi-view fusion network (DGAT-MFN). This model is adept at pinpointing precise attention features by integrating a double-hop graph with the graph attention network (GAT), thereby enhancing the aggregation of multi-level node information and surmounting the limitations of a restricted receptive field. Further, the spectralcoordinate attention module (SCAM) is presented to seize more nuanced spectral and spatial attention features. SCAM harnesses coordinate attention mechanisms for in-depth pixel-level global spectral-spatial view. Coupled with the multi-scale Gabor texture view, we forge a multi-view fusion network (MFN) that meticulously highlights edge details across varying scales and captures beneficial features. Our experimental validation across four renowned benchmark HSI datasets showcases our model's superiority, outstripping comparative methods in classification accuracy with limited labeled samples.
15 February 2023 in Signal, Image and Video Processing
Hyperspectral target detection (HTD) faces significant challenges due to the scarcity of labeled samples and the redundancy in spectral information. To address these challenges, this paper proposes a multi-scale lightweight transformer-based detector (MSLTD) specifically designed for hyperspectral target detection. First, a novel training sample generation strategy is developed that relies solely on raw hyperspectral data and a single prior target spectrum to generate a sufficient number of high-quality training samples. Second, a multi-scale feature lightweight network based on three-channel transformer is proposed. It first divides spectral pixels into consecutive groups to extract local spectral features across adjacent bands, and then introduces a multi-scale sparse subspace transformer to capture and fuse intrinsic spectral characteristics and long-range dependencies at multiple resolutions while effectively reducing spectral redundancy. In addition, an adaptive hardness-aware contrastive (AHC) loss function is introduced, which integrates triplet loss and focal loss to adaptively emphasize hard-to-classify samples and improve the discriminative ability between targets and backgrounds. The effectiveness of the proposed MSLTD is validated through comparative experiments on four real hyperspectral datasets against seven state-of-the-art target detection methods, demonstrating improved detection accuracy and competitive computational efficiency. Extensive ablation studies further confirm the contributions of key components in the framework, highlighting its robustness and applicability in various hyperspectral target detection scenarios. The code for this work is available at https://github.com/2474109721ct/MSLTD for reproducibility purposes.