In this study, we develop a novel deep hierarchical vision transformer (DHViT) architecture for hyperspectral and light detection and ranging (LiDAR) data joint classification. Current classification methods have limitations in heterogeneous feature representation and information fusion of multi-modality remote sensing data (e.g., hyperspectral and LiDAR data), these shortcomings restrict the collaborative classification accuracy of remote sensing data. The proposed deep hierarchical vision transformer architecture utilizes both the powerful modeling capability of long-range dependencies and strong generalization ability across different domains of the transformer network, which is based exclusively on the self-attention mechanism. Specifically, the spectral sequence transformer is exploited to handle the long-range dependencies along the spectral dimension from hyperspectral images, because all diagnostic spectral bands contribute to the land cover classification. Thereafter, we utilize the spatial hierarchical transformer structure to extract hierarchical spatial features from hyperspectral and LiDAR data, which are also crucial for classification. Furthermore, the cross attention (CA) feature fusion pattern could adaptively and dynamically fuse heterogeneous features from multi-modality data, and this contextual aware fusion mode further improves the collaborative classification performance. Comparative experiments and ablation studies are conducted on three benchmark hyperspectral and LiDAR datasets, and the DHViT model could yield an average overall classification accuracy of 99.58%, 99.55%, and 96.40% on three datasets, respectively, which sufficiently certify the effectiveness and superior performance of the proposed method.

Hyperspectral image classification has become a research focus in recent literature. However, well-designed features are still open issues that impact on the performance of classifiers. In this paper, a novel supervised deep feature extraction method based on siamese convolutional neural network (S-CNN) is proposed to improve the performance of hyperspectral image classification. First, a CNN with five layers is designed to directly extract deep features from hyperspectral cube, where the CNN can be intended as a nonlinear transformation function. Then, the siamese network composed by two CNNs is trained to learn features that show a low intraclass and high interclass variability. The important characteristic of the presented approach is that the S-CNN is supervised with a margin ranking loss function, which can extract more discriminative features for classification tasks. To demonstrate the effectiveness of the proposed feature extraction method, the features extracted from three widely used hyperspectral data sets are fed into a linear support vector machine (SVM) classifier. The experimental results demonstrate that the proposed feature extraction method in conjunction with a linear SVM classifier can obtain better classification performance than that of the conventional methods.

Convolutional neural network (CNN) for hyperspectral image classification can provide excellent performance when the number of labeled samples for training is sufficiently large. Unfortunately, a small number of labeled samples are available for training in hyperspectral images. In this letter, a novel semi-supervised convolutional neural network is proposed for the classification of hyperspectral image. The proposed network can automatically learn features from complex hyperspectral image data structures. Furthermore, skip connection parameters are added between the encoder layer and decoder layer in order to make the network suitable for semi-supervised learning. Semi-supervised method is adopted to solve the problem of limited labeled samples. Finally, the network is trained to simultaneously minimize the sum of supervised and unsupervised cost functions. The proposed network is conducted on a widely used hyperspectral image data. The experimental results demonstrate that the proposed approach provides competitive results to state-of-the-art methods.

In recent years, deep-learning-based hyperspectral image (HSI) classification methods have achieved significant development. The superior capability of feature extraction from these data-driven methods dramatically improves the classification performance. However, the previous methods usually require to retrain the network from scratch to obtain the capability of feature extraction adaptive for the target image when facing a new HSI to be classified, which is a time-consuming and redundant process. In this paper, we consider putting this process ahead and making the network have a robust capability of feature extraction with generalization through pre-training. Therefore, the network enables to directly extract features of the target HSI without re-training. For this purpose, we rethink the three-dimension (3D) HSI data from a perspective of spectral sequence, and we attempt to extract the spectral variation information as the spectrum motion feature. Then, we construct an unsupervised spectrum motion feature learning framework (SMF-UL), which can be pre-trained on mass unlabeled HSI data to learn the knowledge about perceiving spectral variation. Furthermore, to achieve the expansion of source data for pre-training, we develop an extendable training dataset construction method, which can integrate HSIs of different sizes, number of bands and sensors into a unified training set to utilize the rapidly growing mass unlabeled HSI data effectively. Finally, we use the trained network to directly extract the spectrum motion feature of the target HSI for classification, so the laborious re-training of the network can be avoided. Extensive experiments show that the proposed SMF-UL acquires the robust capability of feature extraction with generalization through unsupervised learning on mass unlabeled HSI data, and the classification performance of extracted spectrum motion feature is competitive to advanced in-domain and cross-domain methods, which shows its flexibility and superiority. The code of SMF-UL will be open at: https://github.com/sssssyf/SMF-UL.

Recently, the field of hyperspectral image (HSI) classification is dominated by deep learning-based methods. However, training deep learning models usually needs a large number of labeled samples to optimize thousands of parameters. In this article, a deep multiview learning method is proposed to deal with the small sample problem of HSI. First, two views of an HSI scene are constructed by applying principal component analysis to different bands. Second, a deep residual network is designed to embed the different views of a sample to a latent space. The designed deep residual network is trained by maximizing agreement between differently augmented views of the same data sample via a contrastive loss in the latent space. Note that the training procedure of the designed deep residual network does not use labeled information. Therefore, the proposed method belongs to the category of unsupervised learning, which could alleviate the lack of labeled training samples. Finally, a conventional machine learning method (e.g., support vector machine) is used to complete the classification task in the learned latent space. To demonstrate the effectiveness of the proposed method, extensive experiments are carried on four widely used hyperspectral data sets. The experimental results demonstrate that the proposed method could improve the classification accuracy with small samples.