In hyperspectral image (HSI) classification, each pixel sample is assigned to a land-cover category. In the recent past, convolutional neural network (CNN)-based HSI classification methods have greatly improved performance due to their superior ability to represent features. However, these methods have limited ability to obtain deep semantic features, and as the layer’s number increases, computational costs rise significantly. The transformer framework can represent high-level semantic features well. In this article, a spectral–spatial feature tokenization transformer (SSFTT) method is proposed to capture spectral–spatial features and high-level semantic features. First, a spectral–spatial feature extraction module is built to extract low-level features. This module is composed of a 3-D convolution layer and a 2-D convolution layer, which are used to extract the shallow spectral and spatial features. Second, a Gaussian weighted feature tokenizer is introduced for features transformation. Third, the transformed features are input into the transformer encoder module for feature representation and learning. Finally, a linear layer is used to identify the first learnable token to obtain the sample label. Using three standard datasets, experimental analysis confirms that the computation time is less than other deep learning methods and the performance of the classification outperforms several current state-of-the-art methods. The code of this work is available at https://github.com/zgr6010/HSI_SSFTT for the sake of reproducibility.

The joint use of multi-source remote sensing data for earth observation missions has drawn much attention. Although the fusion of several data sources can improve the accuracy of land-cover identification, many technical obstacles exist, such as disparate data structures, irrelevant physical characteristics, and a lack of training data. In this article, a novel dual-branch method, consisting of a hierarchical convolutional neural network (CNN) and a transformer network, is proposed for fusing multi-source heterogeneous information and improving joint classification performance. First, by combining CNN with a transformer, the proposed dual-branch network can significantly capture and learn spectral-spatial features from hyperspectral image data and elevation features from light detection and ranging (LiDAR) data. Then, to fuse these two sets of data features, a cross-token attention fusion encoder is designed in a specialty. The well-designed deep hierarchical architecture takes full advantage of the powerful spatial context information extraction ability of CNN and the strong long-range dependency modeling ability of the transformer network based on self-attention mechanism. Four standard datasets are used in experiments to verify the effectiveness of the approach. The experimental results reveal that the proposed framework can perform noticeably better than state-of-the-art methods. The source code of the proposed method will be available publicly at https://github.com/zgr6010/Fusion_HCT.git.

In recent years, convolutional neural networks (CNNs) have achieved remarkable success in hyperspectral image (HSI) classification tasks, primarily due to their outstanding spatial feature extraction capabilities. However, CNNs struggle to capture the diagnostic spectral information inherent in HSI. In contrast, vision transformers (ViTs) exhibit formidable prowess in handling spectral sequence information and exceling at capturing long-range correlations between pixels and bands. Nevertheless, due to the information loss during propagation, some existing transformer-based classification methods struggle to form sufficient spectral-spatial information mixing. To mitigate these limitations, we propose a memory-augmented spectral-spatial transformer (MASSFormer) for HSI classification. Specifically, MASSFormer incorporates two efficacious modules: the memory tokenizer (MT) and the memory-augmented transformer encoder (MATE). The former serves to transform spectral-spatial features into memory tokens for storing prior knowledge. The latter aims to extend traditional multihead self-attention (MHSA) operations by incorporating these memory tokens, enabling ample information blending while alleviating the potential depth decay in the model and consequently improving the model’s classification performance. Extensive experiments conducted on four benchmark datasets demonstrate that the proposed method outperforms state-of-the-art methods. The source code is available at https://github.com/hz63/ MASSFormer for the sake of reproducibility.

The low spatial resolution of hyperspectral images leads to the coexistence of multiple ground objects in a single pixel (called mixed pixels). A large number of mixed pixels in a hyperspectral image hinders the subsequent analysis and application of the image. In order to solve this problem, a novel sparse unmixing method, which considers highly similar patches in nonlocal regions of a hyperspectral image, is proposed in this paper. This method exploits spectral correlation by using collaborative sparsity regularization and spatial information by employing total variation and weighted nonlocal low-rank tensor regularization. To effectively utilize the tensor decomposition, nonlocal similar patches are first grouped together. Then, these nonlocal patches are stacked to form a patch group tensor. Finally, weighted low rank tensor regularization is enforced to constrain the patch group to obtain an estimated low rank abundance image. Experiments on simulated and real hyperspectral datasets validated the superiority of the proposed method in better maintaining fine details and obtaining better unmixing results.

The effective combination of hyperspectral image (HSI) and light detection and ranging (LiDAR) data can be utilized for land cover classification. Recently, deep learning-based classification methods, especially those utilizing Transformer networks, have achieved remarkable success. However, deep learning classification methods for multi-source data still encounter various technical challenges, such as the comprehensive utilization of multi-scale information, the lightweight network design, and the efficient fusion strategies for heterogeneous data. To address these challenges, we propose a novel and efficient deep neural network, namely multi-scale 3D-2D mixed CNN feature extraction and multi-source data lightweight attention-free fusion network (M2FNet) based on CNN and Transformer. Through end-to-end training, this network effectively combines heterogeneous information from multiple sources, leading to improved performance in joint classification. Specifically, M2FNet employs a multi-scale 3D-2D mixed CNN design to extract both the spatial-spectral features of HSI and the depth-based elevation features of LiDAR data. Subsequently, the extracted features are fed into a novel encoder comprising a feature enhancement module, designed with mathematical morphology and a dilated convolutional module derived from the self-attention of the conventional Transformer encoder (DConvformer), which plays a crucial role in integrating multi-source information within the network. The well-designed architecture enables the network to acquire multi-scale depth and high-order features, significantly reducing the number of training parameters. Comparative experimental results and ablation studies demonstrate that M2FNet outperforms other advanced methods. The source code is publicly available at https://github.com/cupid6868/M2FNet.git.

In the convolutional neural network, the precise segmentation of small-scale objects and object boundaries in remote sensing images is a great challenge. As the model gets deeper, low-level features with geometric information and high-level features with semantic information cannot be obtained simultaneously. To alleviate this problem, a successive pooling attention network (SPANet) was proposed. The SPANet mainly consists of ResNet50 as the backbone, successive pooling attention module (SPAM), and feature fusion module (FFM). Specifically, the SPANet uses two parallel branches to extract high-level features by ResNet50 and low-level features by the first 11 layers of ResNet50. Then, both the high- and low-level features are fed to the SPAM, which is mainly composed of a successive pooling operator and a self-attention submodule, for further extracting deeper multiscale and salient features. In addition, the low- and high-level features after the SPAM are fused by the FFM to achieve the complementarity of spatial and geometric information. This fusion module alleviates the problem of the accurate segmentation of object edges. Finally, the high-level features and enhanced low-level features of the two branches are fused to obtain the final prediction results. Experiments show that the proposed SPANet achieves a good segmentation effect compared with other models on two remotely sensed datasets.