21 October 2022 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
The hyperspectral image (HSI) has nearly continuous spectral information, thus, the target of interest can be accurately identified by the subtle details of spectral properties. Spectral resolution at different scales can capture different levels of spectral features: small-scale spectral bands are beneficial for extracting global details in vision transformers, while large-scale spectral bands are more effective for local features. Transformer shows advantages in global information extraction with self-attention module and even surpasses CNNs in various tasks. Some works based on the vision transformer have performed surprisingly in HSI classification. However, single-scale vision transformers are insufficient to balance the extraction of local details and redundancy on different scales. The recent work, a multi-scale vision transformer, has provided a solution with spatial patch-wise features in image classification. Inspired by this, we propose the Cross-spectral vision transformer (CSiT) with two branches to extract pixel-wise multi-scale features and further design a multi-scale spectral embedding module to enhance local details between neighboring spectral bands. Moreover, based on the cross-attention operation, a single token for each branch is recognized as a query and used to exchange information with other branches. We evaluate the classification performance of the proposed CSiT in three classic HSI datasets with extensive experiments, showing the multi-scale vision transformer architecture has a promising result for HSI classification with one-dimensional spectral bands.
4 April 2024 in IEEE Sensors Journal
Accurately segmenting geological hazard areas in high-resolution remote sensing images faces challenges due to the arbitrary scopes and irregular contours of these areas. In this paper, we propose a novel method for abnormal geological region segmentation in high-resolution remote sensing images. Specifically, the proposed method consists of two modules: Channel Transformer Encoder Module (CTEM) and Multi-scale Global Attention Module (MSGAM). On the one hand, MSGAM is designed to effectively utilize low and high-level features. To this end, it fuses multi-scale representations extracted via dilated convolutions with varying dilation rates. To model long-range multi-scale dependencies, CTEM is enabled to automatically assign importance weights to multi-scale features and adapt receptive fields accordingly. In this way, our method can address the limited global context of convolutional operations. The proposed method achieves state-of-the-art results on the dataset of landslide and debris flow disasters. Experimental results demonstrate the effectiveness and superiority of the proposed method in handling irregular contours and shapes of abnormal geological regions.
10 October 2023 in Digital Signal Processing