Hyperspectral imaging (HSI) captures abundant spectral information of land covers while light detection and ranging (LiDAR) provides elevation and structural characteristics. Joint classification of HSI and LiDAR data can effectively merge spectral and elevation information to enhance the outcome of land cover classification. Current HSI and LiDAR joint classification approaches mainly employ a three-layer deep network to extract high-order features, followed by a concatenation or weighted fusion scheme which cannot fully exploit the unique properties of different data modalities. Meanwhile, these methods usually require high computational resources. To alleviate these issues, this paper proposes a masked self-attention fusion network (MSAF) for joint HSI and LiDAR classification, where a cascaded cross-attention fusion framework is designed to fully merge different stages of features. First, a mobile convolution block is developed to extract multi-modal data features. Then, a multi-view sequence embedding method is proposed to effectively integrate elevation information and spectral-spatial information so as to obtain token sequences. Finally, an effective masked self-attention mechanism is designed to fuse token sequences. Experimental results on multiple datasets indicate that the proposed framework significantly outperforms other advanced multi-modal fusion methods in terms of classification performance and computing efficiency. The code of this manuscript is available on https://github.com/lulushh/MSAF.

Recently, deep learning has been widely used in hyperspectral image classification. Reparameterized network as a novel deep learning method achieves competitive performance compared to other methods. Hyperspectral images have the defect of a large amount of data, and reparameterization can improve network performance without increasing network complexity because of the structure fusion. However, according to our survey there is no relevant application in hyperspectral classification. In this paper, we propose a network using reparameterized approach, which is applied in the field of hyperspectral image classification firstly. This network reparameterizes the Spectral-Spatial Residual Network (SSRN), abbreviated as RepSSRN. Experiments in three classical datasets show that RepSSRN has improved the performance of SSRN.

In hyperspectral image classification, using spatial information as a supplement to spectral information is an effective way to improve classification accuracy. In this paper, a novel robust complementary method using spectral–spatial information is proposed to reduce the information loss in feature extraction, thus improving the classification effect. In short, two complementary feature extraction stages are used to get the probability maps for decision fusion. In the stage of pre-processing feature extraction, we propose an adaptive cubic total variational smoothing method (ACTVSP), which is first proposed and applied in the remote sensing research field, to obtain the first-stage probability map. At the same time, we utilize edge-preserving filtering in the post-processing stage and obtain the second probability map by the means of pixel-level classifier. Finally, the probability-like maps obtained in the above two stages are integrated by decision fusion rules. Experiments on ten public data sets show that the effectiveness of our proposed method and demonstrate the superior of distinguishing different land covers on the basis of very few training samples. Therefore, it can be applied to practical applications in different scenes.

Effective feature extraction (FE) has always been the focus of hyperspectral images (HSIs). For aerial remote-sensing HSIs processing and its land cover classification, in this article, an efficient two-staged hyperspectral FE method based on total variation (TV) is proposed. In the first stage, the average fusion method was used to reduce the spectral dimension. Then, the anisotropic TV model with different regularization parameters was utilized to obtain featured blocks of different smoothness, each containing multi-scale structure information, and we stacked them as the next stage’s input. In the second stage, equipped with singular value transformation to reduce the dimension again, we followed an isotropic TV model based on split Bregman algorithm for further detail smoothing. Finally, the feature-extracted block was fed to the support vector machine for classification experiments. The results, with three hyperspectral datasets, demonstrate that our proposed method can competitively outperform state-of-the-art methods in terms of its classification accuracy and computing time. Also, our proposed method delivers robustness and stability by comprehensive parameter analysis.