Deep learning-based hyperspectral image super-resolution (SR) methods have achieved remarkable success, which can improve the spatial resolution of hyperspectral images with abundant spectral information. However, most of them utilize 2D or 3D convolutions to extract local features while ignoring the rich global spatial-spectral information. In this paper, we propose a novel method called the Multi-Scale Deformable Transformer (MSDformer) for single hyperspectral image super-resolution (SR). The proposed method incorporates the strengths of the convolutional neural network for local spatial-spectral information and the Transformer structure for global spatial-spectral information. Specifically, a multi-scale spectral attention module based on dilated convolution is designed to extract local multi-scale spatial-spectral information, which leverages shared module parameters to exploit the intrinsic spatial redundancy and spectral attention mechanism to accentuate the subtle differences between different spectral groups. Then a deformable convolution-based Transformer module is proposed to further extract the global spatial-spectral information from the local multi-scale features of the previous stage, which can explore the diverse long-range dependencies among all spectral bands. Extensive experiments on three hyperspectral datasets demonstrate that the proposed method achieves excellent SR performance and outperforms the state-of-the-art methods in terms of quantitative quality and visual results. The code is available at https://github.com/Tomchenshi/MSDformer.git.
15 October 2024 in IEEE Transactions on Image Processing
Hyperspectral image super-resolution has attained widespread prominence to enhance the spatial resolution of hyperspectral images. However, convolution-based methods have encountered challenges in harnessing the global spatial-spectral information. The prevailing transformer-based methods have not adequately captured the long-range dependencies in both spectral and spatial dimensions. To alleviate this issue, we propose a novel cross-scope spatial-spectral Transformer (CST) to efficiently investigate long-range spatial and spectral similarities for single hyperspectral image super-resolution. Specifically, we devise cross-attention mechanisms in spatial and spectral dimensions to comprehensively model the long-range spatial-spectral characteristics. By integrating global information into the rectangle-window self-attention, we first design a cross-scope spatial self-attention to facilitate long-range spatial interactions. Then, by leveraging appropriately characteristic spatial-spectral features, we construct a cross-scope spectral self-attention to effectively capture the intrinsic correlations among global spectral bands. Finally, we elaborate a concise feed-forward neural network to enhance the feature representation capacity in the Transformer structure. Extensive experiments over three hyperspectral datasets demonstrate that the proposed CST is superior to other state-of-the-art methods both quantitatively and visually. The code is available at https://github.com/Tomchenshi/CST.git.
16 September 2024 in IEEE Transactions on Circuits and Systems for Video Technology
Integrating low-resolution hyperspectral images with high-resolution multispectral images is an effective approach to derive high-resolution hyperspectral images. Recently, numerous deep learning-based approaches have been employed to model the mapping relationships for the fusion directly. However, these methods often neglect the spectral characteristics and fail to facilitate comprehensive interactions among global features from heterogeneous modalities. In this paper, we propose a novel cyclic Transformer based on the cross-modality spatial-spectral interaction, exploiting diverse interaction modes to explore the similarity and complementarity among cross-modality features. Specifically, we design a cyclic interactive architecture to fully exploit the abundant spectral prior information in low-resolution hyperspectral images and the rich spatial prior information in high-resolution multispectral images. By incorporating spatial and spectral priors into the attention mechanisms in Transformer modules, we explore the long-range dependency information within the cross-modality features. Furthermore, to enhance interaction among features from different modalities, we devise the cross-modality adaptive interaction mechanisms in both spatial and spectral dimensions to facilitate information reciprocity between different modalities. Extensive experiments demonstrate that the proposed approach outperforms the state-of-the-art fusion methods both quantitatively and visually. The code is available at https://github.com/Tomchenshi/CYformer.
30 October 2025 in IEEE Geoscience and Remote Sensing Letters
Although Mamba-based models have demonstrated great potential in hyperspectral image (HSI) classification, existing approaches often rely on patch-wise inputs, causing redundant computation and limiting global spectral–spatial modeling, while the absence of hierarchical representation and explicit interaction further constrains fine-grained fusion. To address these limitations, we propose a Progressive Hybrid Mamba Model (PHDMamba), which is designed to progressively capture long-range spectral-spatial contextual dependencies and gradually fuse spectral and spatial information through adaptive feature interaction, ultimately yielding a unified representation. Specifically, we develop a Progressive Hybrid Mamba Module, which performs stage-wise modeling of long-range dependencies along both spectral and spatial dimensions. In addition, a dedicated Spectral-Spatial Interaction Module is introduced to adaptively integrate contextual spectral and spatial features. Experimental results on three widely used HSI benchmark datasets demonstrate that the proposed method achieves superior classification performance compared to existing approaches. The implementation code will be released publicly.
Mamba has demonstrated exceptional performance in visual tasks due to its powerful global modeling capabilities and linear computational complexity, offering considerable potential in hyperspectral image super-resolution (HSISR). However, in HSISR, Mamba faces challenges as transforming images into 1D sequences neglects the spatial-spectral structural relationships between locally adjacent pixels, and its performance is highly sensitive to input order, which affects the restoration of both spatial and spectral details. In this paper, we propose HSRMamba, a contextual spatial-spectral modeling state space model for HSISR, to address these issues both locally and globally. Specifically, a local spatial-spectral partitioning mechanism is designed to establish patch-wise causal relationships among adjacent pixels in 3D features, mitigating the local forgetting issue. Furthermore, a global spectral reordering strategy based on spectral similarity is employed to enhance the causal representation of similar pixels across both spatial and spectral dimensions. Finally, experimental results demonstrate our HSRMamba outperforms the state-of-the-art methods in quantitative quality and visual results. Code is available at: https://github.com/Tomchenshi/HSRMamba.
In image super‐resolution, deep neural networks with various attention mechanisms have achieved noticeable performance in recent years, for example, channel attention and layer attention. Although many researchers have achieved good super‐resolution results with only a certain style of attention, the divergence and the complementarity focused by multiple attention mechanisms are ignored. In addition, most of these methods fail to utilize the diverse information from multi‐scale features. To efficiently manipulate the above rich information, this paper strives to combine multi‐scale structure and multi‐attention schemes in architecture and module levels for super‐resolution. Especially, in the architecture level, a fused pyramid attention network is developed to extract deep features with the multi‐scale context information from multiple different sizes of receptive field recurrently with skip connections. For the module level, a fused pyramid attention module is designed to fuse the two attention mechanisms to further refine the deep features with fine‐grained information. Compared with the common fusion strategy, the adopted feature fusion structure can maintain better structural information while establishing long‐range dependency. Extensive experimental results demonstrate that the proposed network achieves favorable performance quantitatively and visually.