The rapid development of sensor and multimodal technology has provided more possibilities for multi-source remote sensing image classification. However, some existing joint classification methods are limited to single-level feature fusion and fail to fully explore the deep correlation between cross-level features, thus limiting the effective interaction and complementarity of information between different modal data. To alleviate this issue, this paper proposes a hierarchical multi-modal feature aggregation based multi-head axial attention transformer (HMAT) for joint classification of Hyperspectral and LiDAR data. Firstly, a hierarchical multimodal feature aggregation module (HMFA) is proposed to more effectively fuse spatial-spectral features of HSIs and elevation features of LiDAR data and generate more discriminative low-dimensional feature representations. Secondly, a pyramid-inverted pyramid convolution module (PIP) is designed. Through the complementary feature extraction structure, PIP can more fully capture the multi-scale local features in the fused feature map of hyperspectral and LiDAR data. Finally, a multi-head axial attention (MHAA) component is constructed to capture information at different scales in the fused feature maps, thereby accurately modeling global dependencies. The proposed HMAT has been extensively tested on three publicly available datasets. Experimental results demonstrate that the classification performance of the proposed method outperforms that of several state-of-the-art methods.

Convolutional neural networks (CNNs) have been widely used in hyperspectral image classification due to their ability to extract image features effectively. However, under the condition of limited samples, the modeling ability of CNNs for the relationships among samples is limited. At present, research on the classification of hyperspectral images with a small number of samples remains an important challenge in the field of hyperspectral image processing. Recently, graph convolutional networks (GCNs) have been applied in hyperspectral image classification tasks. In this paper, a multi-hop graph rectifies attention and spectral overlap grouping convolutional fusion network (MRSGFN) for hyperspectral image classification is proposed. In the graph convolution branch, a multi-hop graph rectify attention (MHRA) is designed to weight and correct the features extracted by graph convolution. In the convolutional branch, to solve the problem of dimensionality disaster caused by high spectral dimension with a small number of samples, a spectral intra-inter group feature extraction module (SI2FEM) based on spectral overlap grouping is constructed. In order to better fuse the features extracted from CNNs and GCNs, a gaussian weighted fusion module (GWFM) is elaborately designed in this paper. The features extracted by different branches are assigned different weights by GWFM through a two-dimensional Gaussian map and then fused. Numerous experiments were conducted on three common datasets and showed that the classification performance of the proposed MRSFGN is superior to other advanced methods.

Multi-source remote sensing image classification based on Mamba has received increasing attention. However, existing methods do not consider the non-causal characteristics of visual data, which leads to insufficient extraction of global features by the model. To alleviate this problem, a novel network called TBi-Mamba is proposed for joint classification of hyperspectral and LiDAR data. Firstly, a cross-modal knowledge search module (CMKS) is designed, which effectively captures local features in different modalities through multi-scale feature extraction and interaction between multi-modal data. Secondly, a triple bidirectional sequence scanning mamba module (TBi-M) is proposed, which comprehensively considers multimodal information from the perspective of bidirectional sequence scanning, and introduces Mamba to accurately model global dependencies. Finally, a mixed feature reconstruction module (MFRM) is constructed. This module constructs an auxiliary loss function by reconstructing images of different modalities, providing more comprehensive supervision information and thus improving the performance of the model. The proposed method was evaluated on three publicly available datasets, and experimental results fully demonstrated that the classification performance of the proposed method is superior to that of some state-of-the-art (SOTA) methods.

In recent years, convolutional neural network (CNN) based methods have been widely used in remote sensing image compression tasks. However, CNN is commonly used to extract local information and does not fully utilize global contextual information. The Transformer model can effectively extract the latent contextual information in remote sensing images due to its multi-head self-attention mechanisms. Due to the multi-scale local features and global low-frequency information of remote sensing images, existing deep learning-based compression methods have not effectively combined CNN and Transformer. In order to overcome the limitations of the above methods, a multi-head global attention and spatial spectral information fusion network (MGSSNet) is proposed for remote sensing image compression. First, a spatial spectral information fusion attention module (SSIF-AM) is constructed to obtain multi-scale local information. Secondly, a multi-head global attention module (MHG-AM) is proposed to capture rich global context information. Thirdly, a local-global collaboration module (LGCM) is developed to explore the correlation between the multi-scale local features obtained by SSIF-AM and the global visual features obtained by MHG-AM, and to efficiently model the intrinsic relationships between them to achieve effective feature fusion. Experimental results show that compared with advanced compression models, the proposed MGSSNet method achieves better compression performance. In addition, using reconstructed images obtained by different compression methods for classification tasks has proven that the proposed method can help achieve better classification performance, indicating that the proposed compression method can more fully preserve important information in the image.

For hyperspectral image classification (HSIC), labeling samples is challenging and expensive due to high dimensionality and massive data, which limits the accuracy and stability of classification. To alleviate this problem, a greedy strategy guided graph self-attention network (GS-GraphSAT) is proposed. Firstly, a graph self-attention mechanism (GSA) is designed by combining multi-head self-attention mechanism with graph attention network, which can simultaneously consider the direct and indirect relationships between nodes, and deeply analyze the intrinsic characteristics of nodes. Secondly, a multi-attention fusion module (MAF) is developed, which utilizes multi-scale convolution kernels and attention mechanisms to significantly enhance the network’s ability to extract local features from images at the pixel level, thereby further enriching the hierarchy and diversity of features. Finally, a greedy training strategy (GTS) is proposed. During the training process, GTS accurately determines the optimal time to supplement samples by analyzing the changes in losses, thereby achieving a significant improvement in network classification performance with limited samples. Extensive experiments were conducted on four challenging datasets. The results demonstrate that the proposed method significantly outperforms other state-of-the-art methods in terms of classification accuracy and robustness. The performance improvement of OA can reach up to 1.70% on Houston 2013. The codes of this work will be available at https://github.com/Isee-max/IEEE_TGRS_GS-GraphSAT for reproduction.