RGB image spectral super-resolution (SSR) is a challenging task due to its serious ill-posedness, which aims at recovering a hyperspectral image (HSI) from a corresponding RGB image. In this article, we propose a novel hybrid 2-D-3-D deep residual attentional network (HDRAN) with structure tensor constraints, which can take fully advantage of the spatial-spectral context information in the reconstruction progress. Previous works improve the SSR performance only through stacking more layers to catch local spatial correlation neglecting the differences and interdependences among features, especially band features; different from them, our novel method focuses on the context information utilization. First, the proposed HDRAN consists of a 2D-RAN following by a 3D-RAN, where the 2D-RAN mainly focuses on extracting abundant spatial features, whereas the 3D-RAN mainly simulates the interband correlations. Then, we introduce 2-D channel attention and 3-D band attention mechanisms into the 2D-RAN and 3D-RAN, respectively, to adaptively recalibrate channelwise and bandwise feature responses for enhancing context features. Besides, since structure tensor represents structure and spatial information, we apply structure tensor constraint to further reconstruct more accurate high-frequency details during the training process. Experimental results demonstrate that our proposed method achieves the state-of-the-art performance in terms of mean relative absolute error (MRAE) and root mean square error (RMSE) on both the ``clean'' and ``real world'' tracks in the NTIRE 2018 Spectral Reconstruction Challenge. As for competitive ranking metric MRAE, our method separately achieves a 16.06% and 2.90% relative reduction on two tracks over the first place. Furthermore, we investigate HDRAN on the other two HSI benchmarks noted as the CAVE and Harvard data sets, also demonstrating better results than state-of-the-art methods.

A new framework is proposed for the fusion of hyperspectral and light detection and ranging (LiDAR) data based on the extinction profiles (EPs), local binary pattern (LBP), and kernel collaborative representation classification. Specifically, EP and LBP features are extracted from both sources. Then, the derived features of each source are classified by collaborative representation-based classifier with Tikhonov regularization (KCRT). Reconstruction residuals are fused to produce the final label assignment. Experimental results demonstrate that the proposed method outperforms the existing methods in hyperspectral and LiDAR data fusion.

This article presents a deep residual network-based fusion framework for hyperspectral and LiDAR data. In this framework, three new fusion methods are proposed, which are the residual network-based deep feature fusion (RNDFF), the residual network-based probability reconstruction fusion (RNPRF) and the residual network-based probability multiplication fusion (RNPMF). The three methods use extinction profile (EP), local binary pattern (LBP), and deep residual network. Specifically, EP and LBP features are extracted from two sources and stacked as spatial features. For RNDFF, the deep features of each source are extracted by a deep residual network, and then the deep features are stacked to create the fusion features which are classified by softmax classifier. For RNPRF, the deep features of each source are input to the softmax classifier to obtain the probability matrices, and then the probability matrices are fused by weighted addition to producing the final label assignment. For RNPMF, the probability matrices are fused by array multiplication. Experimental results demonstrate that the classification performance of the proposed methods significantly outperform existing methods in hyperspectral and LiDAR data fusion.

Recently, deep neural network (DNN)-based methods have achieved good results in terms of the fusion of low spatial resolution hyperspectral (LR HS) and high spatial resolution multispectral (HR MS) images. However, the spectral band correlation (SBC) and the spatial nonlocal similarity (SNS) in hyperspectral (HS) images are not sufficiently exploited by them. To model the two priors efficiently, we propose a spectral-spatial dual graph unfolding network (SDGU-Net), which is derived from the optimization of graph regularized restoration models. Specifically, we introduce spectral and spatial graphs to regularize the reconstruction of the desired high spatial resolution hyperspectral (HR HS) image. To explore the SBC and SNS priors of HS images in feature space and utilize the powerful learning ability of DNNs simultaneously, the iterative optimization of the spectral and spatial graph regularized models is unfolded as a network, which is composed of spectral and spatial graph unfolding modules. The two kinds of modules are designed according to the solutions of the spectral and spatial graph regularized models. In these modules, we employ graph convolution networks (GCNs) to capture the SBC and SNS in the fused image. Then, the learned features are integrated by the corresponding feature fusion modules and fed into the feature condense module to generate the HR HS image. We conduct extensive experiments on three benchmark datasets and the results demonstrate the effectiveness of our proposed SDGU-Net.

A new method of superpixel segmentation-based local pixel neighborhood preserving embedding (SSLPNPE) is proposed for the fusion of hyperspectral and light detection and ranging (LiDAR) data based on the extinction profiles (EPs), superpixel segmentation and local pixel neighborhood preserving embedding (LPNPE). A new workflow is proposed to calibrate the Goddard’s LiDAR, hyperspectral and thermal (G-LiHT) data, which allows our method to be applied to actual data. Specifically, EP features are extracted from both sources. Then, the derived features of each source are fused by the SSLPNPE. Using the labeled samples, the final label assignment is produced by a classifier. For the open standard experimental data and the actual data, experimental results prove that the proposed method is fast and effective in hyperspectral and LiDAR data fusion.

This article presents a label similarity probability filter (LSPF) to make hyperspectral image postclassification. The LSPF is inspired by the first law of geography and proposes a class label probability function to quantify the probability of both centered and its neighboring pixels belonging to the same class. It first classifies the hyperspectral data using the regular support vector machine classifier. Then, it binarizes the posterior classification result to obtain the binary label maps of each class. After that, it traverses all spatial windows centered by each pixel and calculates the cumulative probability of all pixels in each class. Finally, the cumulative probabilities are used to make reclassification to obtain the refined classification map. The experiments on Indian Pines, Pavia University, and ZY1-02D Yellow River Estuary data show that LSPF greatly improves the classification accuracy of spectral signatures and outperforms other state-of-the-art spectral–spatial methods.