Hyperspectral image (HSI) classification is an active research topic in remote sensing. Supervised learning-based methods have been widely used in HSI classification tasks due to their powerful feature extraction capabilities for cases of sufficiently labeled samples. However, practical applications often have limited samples with accurate labels due to the high cost of labeling or unreliable visual interpretation. We introduce a contrastive self-supervised learning (SSL) algorithm to achieve HSI classification for problems with few labeled samples. First, a new HSI-specific augmentation module is developed to generate sample pairs. Then, a contrastive SSL model based on Siamese networks is used to extract features from these easily accessible sample pairs. Finally, the labeled samples are taken to fine-tune the parameters of the classification model to boost classification performance. Tests of the contrastive self-supervised algorithm have been performed on two widely used HSI datasets. The experimental results reveal that the proposed algorithm requires a few labeled samples to achieve superior performance.

The transformer framework has shown great potential in the field of hyperspectral image (HSI) classification due to its superior global modeling capabilities when compared to convolutional neural networks (CNNs). To utilize the transformer to model spatial-spectral information, a hybrid transformer that integrates multi-granularity tokens and spatial-spectral attention is proposed. Specifically, a token generator is designed to embed the multi-granularity semantic tokens, which contribute richer image features to the model by exploiting the CNN’s local representation capability. Moreover, a transformer encoder with a spatial-spectral attention mechanism is proposed to capture the global dependencies between different tokens, enabling the model to focus on more differentiated channels and spatial locations to improve the classification accuracy. Ultimately, adaptive weighted fusion is applied to different granularity transformer branches to boost HybridFormer’s classification performance. Experiments were conducted on four new challenging datasets, and the results indicate that HybridFormer achieves state-of-art results in terms of classification performance. The code of this work will be available at https://github.com/zhaolin6/HybridFormer for the sake of reproducibility.

The joint sparse representation (JSR)-based classifier assumes that pixels in a local window can be jointly and sparsely represented by a dictionary constructed by the training samples. The class label of each pixel can be decided according to the representation residual. However, once the local window of each pixel includes pixels from different classes, the performance of the JSR classifier may be seriously decreased. Since correlation coefficient (CC) is able to measure the spectral similarity among different pixels efficiently, this letter proposes a new classification method via fusing CC and JSR, which attempts to use the within-class similarity between training and test samples while decreasing the between-class interference. First, the CCs among the training and test samples are calculated. Then, the JSR-based classifier is used to obtain the representation residuals of different pixels. Finally, a regularization parameter λ is introduced to achieve the balance between the JSR and the CC. Experimental results obtained on the Indian Pines data set demonstrate the competitive performance of the proposed approach with respect to other widely used classifiers.

Hyperspectral image (HSI) data have large numbers of bands that probably not all bands are equally informative and predictive for an effective HSI classification. Effective algorithms are highly desired in many real-world HSI applications, especially in cases requiring rapid learning with limited computing power. To address the abovementioned case, we present in this article a novel plug-and-play compact band weighting (CBW) module based on the attention-driven mechanism that evaluates different spectral bands according to their contributions to a given classification task. Compared to existing band weighting (BW) modules with tens of thousands of network parameters by deep learning, the proposed CBW is a lightweight module with only 20 parameters. Both model complexity and time cost are significantly reduced. The CBW module implements BW by making full use of the correlation among the adjacent spectral bands and spectral statistic information and, thereby, leads to the effect of recalibrated HSI. The experimental study has been conducted on three widely used HSI data sets, and results show the superiority of the proposed algorithm over current state-of-the-art methods of BW. The source code is available at https://github.com/JarvenYi/CBW .

Visual object tracking plays a fundamental role in computer vision. Extracting the unique spectral and spatial features of hyperspectral images (HSIs) can significantly improve tracking performance in complex scenarios, especially in hyperspectral object tracking. However, due to the limited training samples, handcrafted features are employed in most current hyperspectral trackers, although they cannot sufficiently describe the intrinsic nature of the object. This letter proposes a band regrouping and response-level fusion network (BRRF-Net) for hyperspectral object tracking based on deep transfer learning, employing a deep model trained on color videos to represent features to solve this problem. Specifically, a new band regrouping subnetwork that generates the band weights using hyperspectral feature information is proposed. The bands are divided into several groups using band weights and imported into the Siamese network. Finally, the response-level fusion strategy is adopted to integrate the tracker results for the precise location of objects. Experiments on hyperspectral video reveal that the accuracy of the BRRF-Net is up to 0.689, which is the state-of-the-art performance compared with the current hyperspectral object trackers and proves the effectiveness and superiority of the BRRF-Net.

The superior local context modelling capability of convolutional neural networks (CNNs) in representing features allows greatly enhanced performance in hyperspectral image (HSI) classification tasks by CNN-based methods. However, most of these methods suffer from a restricted receptive field and poor performance in the continuous data domain. To address these issues, we propose a multi-granularity vision transformer via semantic token (MSTViT) for HSI classification, which differs from the existing transformer view by modelling the HSI classification tasks as word embedding problems. Specifically, the MSTViT model extracts multi-level semantic features by a ladder feature extractor and applies a multi-granularity patch embedding module to embed these features simultaneously as different-scale tokens. Moreover, different-granularity tokens are fed to the vision transformer to capture the long-distance dependencies among the different tokens. A depth-wise separable convolution multi-layer perceptron is used to assist the attention mechanism for further excavation of the deep information of HSI. Finally, the performance of HSI classification is improved by fusing the coarse- and fine-granularity representations to generate stronger features. Experimental results on four standard datasets verify the marked improvement of the MSTViT over state-of-the-art CNN and transformer structures. The code of this work is available at https://github.com/zhaolin6/MSTViT for the sake of reproducibility.