Hyperspectral images (HSIs) contain spatially structured information and pixel-level sequential spectral attributes. The continuous spectral features contain hundreds of wavelength bands and the differences between spectra are essential for achieving fine-grained classification. Due to the limited receptive field of backbone networks, convolutional neural networks (CNNs)-based HSI classification methods show limitations in modeling spectral-wise long-range dependencies with fixed kernel size and a limited number of layers. Recently, the self-attention mechanism of transformer framework is introduced to compensate for the limitations of CNNs and to mine the long-term dependencies of spectral signatures. Therefore, many joint CNN and Transformer architectures for HSI classification have been proposed to obtain the merits of both networks. However, these architectures make it difficult to capture spatial–spectral correlation and CNNs distort the continuous nature of the spectral signature because of the over-focus on spatial information, which means that the transformer can easily encounter bottlenecks in modeling spectral-wise similarity and long-range dependencies. To address this problem, we propose a neighborhood enhancement hybrid transformer (NEHT) network. In particular, a simple 2D convolution module is adopted to achieve dimensionality reduction while minimizing the distortion of the original spectral distribution by stacked CNNs. Then, we extract group-wise spatial–spectral features in a parallel design to enhance the representation capability of each token. Furthermore, a feature fusion strategy is introduced to increase subtle discrepancies of spectra. Finally, the self-attention of transformer is employed to mine the long-term dependencies between the enhanced feature sequences. Extensive experiments are performed on three well-known datasets and the proposed NEHT network shows superiority over state-of-the-art (SOTA) methods. Specifically, our proposed method outperforms the SOTA method by 0.46%, 1.05% and 0.75% on average in overall accuracy, average accuracy and kappa coefficient metrics.

Inspired by certain optimization solvers, the deep unfolding network (DUN) usually inherits a multi-phase structure for image compressive sensing (CS). However, in existing DUNs, the message transmission within and between phases still faces two issues: 1) the roughness of transmitted information, e.g., the low-dimensional representations. 2) the inefficiency of transmitted policy, e.g., simply concatenating deep features. In this paper, by unfolding the Proximal Gradient Descent (PGD) algorithm, a novel gradient guided DUN (G2 -DUN) for image CS is proposed, in which a gradient map is delicately introduced within each phase for providing richer informational guidance at both intra-phase and inter-phase levels. Specifically, corresponding to the gradient descent (GD) of PGD, a gradient guided GD module is designed, in which the gradient map can adaptively guide step size allocation for different textures of input image, realizing a content-aware gradient updating. On the other hand, corresponding to the proximal mapping (PM) of PGD, a gradient guided PM module is developed, in which the gradient map can dynamically guide the exploring of deep textural priors in multi-scale space, achieving the dynamic perception of the proposed deep model. By introducing the gradient map, the proposed message transmission system not only facilitates the informational communication between different functional modules within each phase, but also strengthens the inferential cooperation among cascaded phases. Extensive experiments manifest that the proposed G2 -DUN outperforms existing state-of-the-art CS methods.

ZY1-02D is a Chinese hyperspectral satellite, which is equipped with a visible near-infrared multispectral camera and a hyperspectral camera. Its data are widely used in soil quality assessment, mineral mapping, water quality assessment, etc. However, due to the limitations of CCD design, the swath of hyperspectral data is relatively smaller than multispectral data. In addition, stripe noise and collages exist in hyperspectral data. With the contamination brought by clouds appearing in the scene, the availability is further affected. In order to solve these problems, this article used a swath reconstruction method of a spectral-resolution-enhancement method using ResNet (SRE-ResNet), which is to use wide swath multispectral data to reconstruct hyperspectral data through modeling mappings between the two. Experiments show that the method (1) can effectively reconstruct wide swaths of hyperspectral data, (2) can remove noise existing in the hyperspectral data, and (3) is resistant to registration error. Comparison experiments also show that SRE-ResNet outperforms existing fusion methods in both accuracy and time efficiency; thus, the method is suitable for practical application.

Hyperspectral images (HSIs) have been widely used in earth observation because they contain continuous and detailed spectral information which is beneficial for the fine-grained diagnosis of the land cover. In the past few years, convolutional neural networks (CNNs) based methods show limitations in modeling spectral-wise long-range dependencies. Recently, Transformer based deep learning methods are proposed and have shown superiority in modeling the continuous representation of the spectral signatures because the self-attention (SA) mechanism has a global receptive field. Due to the special tokenization of the transformer-based methods, the redundant tokens contained in spectral embeddings are always involved in SA operation. Redundant tokens do not positively contribute to classification. Specifically, the overlapped group-wise tokenization approach may aggravate the Hughes phenomenon and impose additional computations. To address this issue, a lightweight Spatial-Spectral Pyramid Transformer (SSPT) framework is proposed to efficiently extract the spatial-spectral features of HSI by progressively reducing redundant tokens in an end-to-end manner. In particular, a token reduction method is proposed to decide which tokens will be involved by computing and comparing token attentiveness between spectral embeddings and the class token. In addition, for those tokens that are defined as redundant information, a token compensation mechanism is proposed to automatically extract supplementary information for classification. Extensive experiments on three standard datasets quantitatively show the superiority of our methods and the ablation experiments qualitatively prove our hypothesis about the feature distribution in transformer architecture.

Masked image modeling (MIM), a common selfsupervised learning (SSL) technique, has been extensively studied for remote sensing image processing. Nevertheless, its effectiveness for hyperspectral imagery (HSI) remains underexplored due to the distinct data structures and high dimensionality. This paper aims to provide a detailed understanding of MIM from different perspectives of representation learning and statistical analysis for HSI classification tasks. Our study reveals that the MIM paradigm injects inductive bias in the attention mechanism of the transformer model, which is advantageous for capturing the local discrepancies between the spectra. We also show that MIM can increase the diversity of the attention heads in every layer, which is beneficial for the model in extracting more discriminative features from different spectral bands. The similarity of representations from various layers further proves this. Furthermore, our investigation highlights how MIM introduces a dynamic perspective to spectral representations, enabling the model to learn more robust and discriminative features. The final numerical experiments indicate that a moderate mask ratio can enhance the performance of downstream tasks. This suggests that designing a more targeted masking strategy might be necessary to achieve higher and more stable gains in downstream task performance. Without bells and whistles, the vanilla MIM improves the overall classification accuracy by an average of 2.69% over its SL counterpart. We hope that our findings can advance the understanding of MIM in HSI and inspire the design of a more stable SSL paradigm for HSI processing.

Hyperspectral image (HSI) classification has advanced significantly through transformer-based models that leverage self-attention (SA) to capture global spectral dependencies. However, most existing HSI transformers implicitly assume that SA provides a faithful and discriminative modeling of spectral information, an assumption that has rarely been examined. We examine SA behavior in HSI classification using adversarial perturbations guided by explainable AI (XAI) techniques. By comparing transformer attention distributions with Integrated Gradients (IG) importance scores, we uncover discrepancies between attended bands and decision-driving features. Experiments show that SA often forms redundant clusters across neighboring spectral bands and that attention patterns evolve across layers, shifting from a localized to a broader focus. Such redundancy improves robustness to spectral corruptions but reduces feature specificity and leads to repeated processing of highly similar bands, increasing computational burden. Our findings question the everyday use of raw attention as an explanatory clue and guide the design of more reliable and interpretable attention mechanisms for HSI.