Convolutional neural networks (CNNs) have been widely used in hyperspectral image (HSI) classification tasks because of their excellent local spatial feature extraction capabilities. However, because it is difficult to establish dependencies between long sequences of data for CNNs, there are limitations in the process of processing hyperspectral spectral sequence features. To overcome these limitations, inspired by the Transformer model, a spatial–spectral transformer with cross-attention (CASST) method is proposed. Overall, the method consists of a dual-branch structures, i.e., spatial and spectral sequence branches. The former is used to capture fine-grained spatial information of HSI, and the latter is adopted to extract the spectral features and establish interdependencies between spectral sequences. Specifically, to enhance the consistency among features and relieve computational burden, we design a spatial–spectral cross-attention module with weighted sharing to extract the interactive spatial–spectral fusion feature intra Transformer block, while also developing a spatial–spectral weighted sharing mechanism to capture the robust semantic feature inter Transformer block. Performance evaluation experiments are conducted on three hyperspectral classification datasets, demonstrating that the CASST method achieves better accuracy than the state-of-the-art Transformer classification models and mainstream classification networks.

Hyperspectral images (HSIs) contain abundant information in the spatial and spectral domains, allowing for a precise characterization of categories of materials. Convolutional neural networks (CNNs) have achieved great success in HSI classification, owing to their excellent ability in local contextual modeling. However, CNNs suffer from fixed filter weights and deep convolutional layers, which lead to a limited receptive field and high computational burden. The recent vision transformer (ViT) models long-range dependencies with a self-attention mechanism and has been an alternative backbone to CNNs traditionally used in HSI classification. However, such transformer-based architectures designate all the input pixels of the receptive field as feature tokens in terms of feature embedding and self-attention, which inevitably limits the ability for learning multiscale features and increases the computational cost. To overcome this issue, we propose a local semantic feature aggregation-based transformer (LSFAT) architecture which allows transformers to represent long-range dependencies of multiscale features more efficiently. We introduce the concept of the homogeneous region into the transformer by considering a pixel aggregation strategy and further propose neighborhood-aggregation-based embedding (NAE) and attention (NAA) modules, which are able to adaptively form multiscale features and capture locally spatial semantics among them in a hierarchical transformer architecture. A reusable classification token is included together with the feature tokens in the attention calculation. In the last stage, a fully connected layer is used to perform classification on the reusable token after transformer encoding. We verify the effectiveness of the NAE and NAA modules compared with the traditional ViT through extensive experiments. Our results demonstrate the excellent classification performance of the proposed method in comparison to other state-of-the-art approaches on several public HSIs.

The multisource remote sensing classification task has two main challenges: 1) How to capture hyperspectral image (HSI) and light detection and ranging (LiDAR) features cooperatively to fully mine the complementary information between data. 2) How to adaptively fuse multisource features, which should not only overcome the imbalance between HSI and LiDAR data but also avoid the generation of redundant information. The Local Information Interaction Transformer (LIIT) model proposed herein can effectively address these above issues. Specifically, multi-branch feature embedding is first performed to help in the fine-grained serialization of multisource features; subsequently, a local-based multisource feature interactor (L-MSFI) is designed to explore HSI and LiDAR features together. This structure provides an information transmission environment for multi-branch features and further alleviates the homogenization processing mode of the self-attention process. More importantly, a multisource feature selection module (MSTSM) is developed to dynamically fuse HSI and LiDAR features to solve the problem of insufficient fusion. Experiments were carried out on three multisource remote-sensing classification datasets, the results of which show that LIIT has more performance advantages than the state-of-the-art CNN and Transformer methods.

In cross-domain hyperspectral image (HSI) classification, the labeled samples of the target domain are very limited, and it is a worthy attention to obtain sufficient class information from the source domain to categorize the target domain classes (both the same and new unseen classes). This article investigates this problem by employing few-shot learning (FSL) in a meta-learning paradigm. However, most existing cross-domain FSL methods extract statistical features based on convolutional neural networks (CNNs), which typically only consider the local spatial information among features, while ignoring the global information. To make up for these shortcomings, this paper proposes novel convolutional transformer-based few-shot learning (CTFSL). Specifically, FSL is first performed in the classes of source and target domains simultaneously to build the consistent scenario. Then, a domain aligner is set up to map the source and target domains to the same dimensions. In addition, a convolutional transformer (CT) network is utilized to extract local-global features. Finally, a domain discriminator is executed subsequently that can not only reduce domain shift, but also distinguish from which domain a feature originates. Experiments on three widely used hyperspectral image datasets indicate that the proposed CTFSL method is superior to the state-of-the-art cross-domain FSL methods and several typical HSI classification methods in terms of classification accuracy.

Deep neural networks play a significant role in hyperspectral image (HSI) processing, yet they can be easily fooled when trained with adversarial samples (generated by adding tiny perturbations to clean samples). These perturbations are invisible to the human eye, but can easily lead to misclassification by the deep learning model. Recent research on defense against adversarial samples in HSI classification has improved the robustness of deep networks by exploiting global contextual information. However, available methods do not distinguish between different classes of contextual information, which makes the global context unreliable and increases the success rate of attacks. To solve this problem, we propose a robust context-aware network able to defend against adversarial samples in HSI classification. The proposed model generates a global contextual representation by aggregating the features learned via dilated convolution, and then explicitly models intraclass and interclass contextual information by constructing a class context-aware learning module (including affinity loss) to further refine the global context. The module helps pixels obtain more reliable long-range dependencies and improves the overall robustness of the model against adversarial attacks. Experiments on several benchmark HSI datasets demonstrate that the proposed method is more robust and exhibits better generalization than other advanced techniques.

This letter introduces a novel spatial-spectral classification method for hyperspectral images (HSIs) based on a structural-kernel collaborative representation (SKCR), which considers one weak assumption of spatial neighborhood that of the pixels in a superpixel belong to the same class when exploiting contextual information in HSI. The proposed method consists of the following steps. First, a superpixel segmentation strategy is used to construct self-adaptive regions for the HSI. Then, the structural information within each superpixel block is extracted based on the density peak and K nearest neighbors. Next, dual kernels are separately utilized for the exploitation of the spectral and the spatial information. Finally, the dual kernels are combined and incorporated into a support-vector-machine classifier. Since the weak assumption of spatial neighborhood is well considered in the collaborative representation, the proposed method showed excellent classification performance for two widely used real hyperspectral data sets even when the number of training samples was relatively small.