Transformer-based networks have gained significant interest in hyperspectral image classification in recent years due to its good performance in processing long sequences and parallel computation. However, the original Transformer still suffer from limitations in capturing or conveying subtle features and modeling spectral sequences. To address these issues and make Transformer more suitable for hyperspectral image classification tasks, this paper introduces Flow Feature Fusion module in Transformer that uses 2-D CNN to connect shallow and deep features and facilitate feature flow between encoders. This augmentation enhances its ability to perceive and process subtle image features. At the same time, Rotary Position Embedding is also introduced in self-attention to model spectral sequences effectively. Building upon these improvements, we propose the FeatureFlow Transformer, a network characterized by its simple structure and superior classification performance. The experimental results are validated on Indian Pines, Pavia University, and Salinas Valley, and comparisons with state-of-the-art models show that our proposed FeatureFlow Transformer achieves notable performance in hyperspectral image classification. The code will be available at https://github.com/xiaolabushiji/FeatureFlow_Transformer_for_Hyperspectral_Image_Classification.
Semantic segmentation is crucial for the precise analysis of remote sensing (RS) images. However, it heavily relies on high-quality data annotation, which is often expensive and labor-intensive to obtain. Interactive image segmentation methods have been introduced to make the annotation of RS data more efficient through user interaction. However, current methods encounter challenges extracting edge features, utilizing user interaction information, and understanding complex scenes. To tackle these issues, we propose a dual-branch network structure named DBCF-Net (Dual-Branch Network and Collaborative Feedback Integration). DBCF-Net consists of a Global Feature Extraction Branch and an Edge Information Enhancement Branch, which effectively integrate global features and edge information. Moreover, we incorporate the Collaborative Feedback Integration mechanism into our framework to effectively utilize user interaction in high-level feature maps. A hybrid training strategy is implemented, involving training on multiple datasets and refinement on specific datasets, enhancing the model's generalization ability. Experiments demonstrate that DBCF-Net achieves competitive performance in interactive RS image segmentation. On the WHU dataset, our method outperforms the RITM model by 10.40% and 38.97% on the NoC@85 and NoC@90 metrics, respectively, indicating a significant improvement. The proposed framework provides a promising solution to alleviate the workload of manual annotations. The code will be available at https://github.com/PengyuHuo/DBCF-Net after the article is accepted.
Hyperspectral remote sensing provides rich spectral information and has been widely used in fine-grained land-cover classification and forest monitoring. However, accurate tree species classification remains challenging due to subtle interspecific spectral differences, similar spatial structures among related species, redundant spectral bands, and the limited ability of existing methods to model long-range spatial–spectral dependencies efficiently. In addition, many existing hyperspectral image classification methods rely on patch-based inputs and sliding-window inference, which often lead to redundant computation and insufficient utilization of global image context. To address these issues, this paper proposes MambaHSINet, a dual-branch bidirectional state space network for full-image pixel-wise hyperspectral tree species classification. Specifically, the proposed network employs a spectral branch and a spatial branch to explicitly extract complementary spectral responses and spatial structural features. A lightweight channel attention mechanism is introduced to emphasize informative spectral–spatial representations while suppressing redundant information. Subsequently, a bidirectional Mamba global modeling module based on selective state space modeling is adopted to capture long-range contextual dependencies in both forward and backward directions with linear computational complexity. Unlike conventional patch-based methods, MambaHSINet takes the entire hyperspectral image as input and produces full-resolution pixel-wise classification maps, thereby avoiding repeated cropping and redundant sliding-window inference. Experimental results on public and self-collected hyperspectral datasets demonstrate that the proposed method achieves a favorable balance among classification accuracy, inference efficiency, and model complexity, showing strong potential for practical tree species classification applications.The source code of this paper will be made available at https://github.com/YFENG-123/mambaHSI after the publication of the paper.
Hyperspectral remote sensing provides rich spectral information and has been widely used in fine-grained land-cover classification and forest monitoring. However, accurate tree species classification remains challenging due to subtle interspecific spectral differences, similar spatial structures among related species, redundant spectral bands, and the limited ability of existing methods to model long-range spatial–spectral dependencies efficiently. In addition, many existing hyperspectral image classification methods rely on patch-based inputs and sliding-window inference, which often lead to redundant computation and insufficient utilization of global image context. To address these issues, this paper proposes MambaHSINet, a dual-branch bidirectional state space network for full-image pixel-wise hyperspectral classification. Specifically, the proposed network employs a spectral branch and a spatial branch to explicitly extract complementary spectral responses and spatial structural features. Subsequently, a bidirectional Mamba global modeling module based on selective state space modeling is adopted to capture long-range contextual dependencies in both forward and backward directions with linear computational complexity. Unlike conventional patch-based methods, MambaHSINet takes the entire hyperspectral image as input and produces full-resolution pixel-wise classification maps, thereby avoiding repeated cropping and redundant sliding-window inference. We also construct two well-annotated subsets of a UAV-borne hyperspectral dataset dedicated to tree species classification. Experimental results on self-collected and public hyperspectral datasets demonstrate that the proposed method achieves excellent classification accuracy, inference efficiency, and generalization performance. It exhibits great potential for practical tree species classification and other general hyperspectral application scenarios.
24 December 2024 in Journal of the Indian Society of Remote Sensing