Hyperspectral image (HSI) classification is a critical task with numerous applications in the field of remote sensing. Although convolutional neural networks have achieved remarkable success in computer vision, they are still limited in the ability to model long-term dependencies due to small receptive fields. Recently, vision transformers have been used in HSI classification, where multi-head self-attention (MHSA), as the key feature extractor of transformers, learns global dependencies in long-range positions and bands of HSI pixels. Existing vision transformers for classifying HSIs with a large number of bands, however, have some limitations in that features extracted by MHSA may exhibit over-dispersion. In this article, we propose a Group-Aware Hierarchical Transformer (GAHT) for HSI classification, which confines MHSA to the local spatial–spectral context by introducing a new grouped pixel embedding (GPE) module. The GPE emphasizes local relationships within HSI spectral channels, resulting in a global–local fashion from a spatial–spectral context for HSI classification. In addition, we construct our transformer in a hierarchical manner, which can significantly improve classification accuracy with only a few parameters. Extensive experiments on four benchmark HSI datasets demonstrate that the proposed method outperforms state-of-the-art HSI classification algorithms. The source code is available at https://github.com/MeiShaohui/Group-Aware-Hierarchical-Transformer .

Convolutional neural networks (CNNs) have been demonstrated to be a powerful tool to automatically learn effective features from large datasets. Though features learned in CNNs are approximately scale-invariant, translation-invariant, and position-invariant, their capacity in dealing with image rotations remains limited. In this paper, a novel cyclic polar coordinate convolutional layer (CPCCL) is proposed for CNNs to handle the problem of rotation-invariance for feature learning. First, the proposed CPCCL converts rotation variation into translation variation using polar coordinates transformation, which can be easily handled by CNNs. Moreover, cyclic convolution is designed to completely handle the translation variation converted from rotation variation by conducting convolution in a cyclic shift mode. Note that the proposed CPCCL is capable of generalization and can be used as a preprocessing layer for classification CNNs to learn the rotation-invariant feature. Extensive experiments over three benchmark datasets demonstrate that the proposed CPCCL can clearly handle the rotation-sensitive problem in traditional CNNs and outperforms several state-of-the-art rotation-invariant feature learning algorithms.

Transformer-based networks, which can well model global characteristics of inputted data using attention mechanism, have been widely applied to hyperspectral image (HSI) classification and achieved promising results. However, existing networks fail to explore complex local land cover structures in different scales of shapes in hyperspectral remote sensing images. Therefore, a novel network named multi-scale and cross-level attention learning (MCAL) network is proposed to fully explore both global and local multi-scale features of pixels for classification. In order to encounter local spatial context of pixels in transformer, a multi-scale feature extraction module (MSFE) is constructed and implemented into transformer-based networks. Moreover, a cross-level feature fusion module (CLFF) is proposed to adaptively fuse features from hierarchical structure of MSFEs using attention mechanism. Finally, the spectral attention module is implemented prior to the hierarchical structure of MSFEs, by which both spatial context and spectral information are jointly emphasized for hyperspectral classification. Experiments over several benchmark datasets demonstrate that the proposed MCAL obviously outperforms both CNN-based and transformer-based state-of-the-art networks for hyperspectral classification.

Emerging vision transformers (ViTs) are more powerful in modeling long-range dependences of features than conventional deep convolution neural networks (CNNs). Thus, they outperform CNNs in several computer vision tasks. However, existing ViTs fail to encounter the multi-scale characteristics of ground objects with various spatial sizes when they are applied to remote sensing (RS) scene images. Therefore, in this paper, a Swin transformer with multi-scale fusion (STMSF) is proposed to alleviate such an issue. Specifically, a multi-scale feature fusion module is proposed, so that features of ground objects at different scales in the RS scene can be well considered by merging multi-scale features. Moreover, a spatial attention pyramid network (SAPN) is designed to enhance the context of coarse features extracted with the transformer and further improve the network’s representation ability of multi-scale features. Experimental results over three benchmark RS scene datasets demonstrate that the proposed network obviously outperforms several state-of-the-art CNN-based and transformer-based approaches.

Transformers have achieved remarkable success in the field of computer vision due to their advantage in capturing the global information of images. However, they fail to model the variance of rotation, resulting in significant performance loss in target detection in remote sensing imagery. In this paper, a rotation-invariant transformer plus model, namely RIFormer+ is proposed to enhance the capabilities of transformers in rotation-invariant feature learning at both long-overlooked local-level and the acknowledged global-level. At the local-level, a rotation-invariant cross-patch embedding (RICPE) module is designed to generate dense patches, which handles encoding inconsistency of tokens with similar semantic information before and after rotation. Moreover, response-enhanced attention (REA) is proposed to extract more rotation-robust global features, which highlights overly dispersed responses ensure sustained attention on discriminative regions. Extensive experiments on three datasets demonstrate the effectiveness of RIFormer+. Without bells and whistles, RIFormer+ increases the classification accuracy by an average of 10% and improves the accuracy on rotated datasets by 20% compared with some state-of-the-art transformers. The code of this paper is available at: https://github.com/psychAo/RIFormerPlus.

Deep learning-based object detection algorithms have achieved significant success in the field of computer vision. However, the wide range of target sizes in remote sensing images poses a challenge for single algorithms to detect objects of varying sizes effectively. To address this issue, this paper proposes an end-to-end object detection algorithm for remote sensing images based on Scale Adaptive and Frequency Fusion DETR (SAFF-DETR), which designs a frequency feature enhancement and fusion mechanism to handle targets of varying sizes within a single framework. First, in order to improve the Transformer-based detectors’ ability to small targets, a multibranch representation fusion (MRF) module is proposed to fuse shallow layer frequency representations, boosting the network’s ability to perceive small targets. Furthermore, Cross-layer Spatial and Channel Frequency Attention (CSFA and CCFA) is designed to enable efficient frequency feature interaction across multi-scale features, enhancing the representation capability for targets of different sizes. Moreover, by integrating the two aforementioned attention mechanisms, the Cross-layer Channel-Spatial-wise Frequency Fusion (CCSFF) structure is introduced to realize global feature interactions in one step without repetitive up and down-sampling operations, by which patch division-based Transformer architecture is designed to enhance scale adaptability for object detection. Experimental results over several benchmark datasets demonstrate that the proposed SAFF-DETR can handle extremely varying-size targets and outperforms several SOTA algorithms. The source code will be available at https://github.com/Jianglin-Zhao/SAFF-DETR.