In multimodal data fusion and land-cover interpretation tasks, the fusion interpretability between hyperspectral image (HSI) and light detection and ranging (LiDAR) data is always nontrivial to be clarified. Furthermore, the heterogeneous sample and distribution variances of these two remote sensing (RS) modalities impede the joint classification performance. In this paper, a Hyperspectral Intrinsic Image Decomposition guided Data Fusion Network (HI ² D ² FNet) is proposed. Generally, classic hyperspectral intrinsic image decomposition (HIID) performs well in image enhancement and shadow removal. It decomposes one HSI into one reflectance component and one shading component. Inspired by the core mechanism of HIID, our motivation is to preliminarily exploit its potential for multimodal RS data fusion and explore the inherent modality connection between HSI and LiDAR data from the intrinsic perspective. Specifically, compared with existing techniques, HI ² D ² FNet is capable of fusing the horizontal geometry information in the shading component with the vertical geometry information in the LiDAR data from both sample and distribution perspectives in the spatial domain. The generated cross-modal geometry feature contributes to guiding the reflectance stream optimization. This unique fusion framework connects both modalities with respect to geometry information and enhances the specific fusion interpretability. The decomposition and fusion modules in HI ² D ² FNet are optimized simultaneously in a novel alternative optimization pattern. Furthermore, several unique cross-modal constraints in terms of prior RS properties are presented. Experiments conducted on three widely available datasets prove the superiority of HI ² D ² FNet over state-of-the-art techniques. The source codes will be available at https://github.com/GEOywb/HI2D2FNet.

A novel deep learning classification method for hyperspectral data based on convolutional neural network is proposed in this paper. Deep learning means bringing multiple layers instead of one to the structure. Through convolution layers and pooling layers, the features in different layers are extracted from original spectral feature images. The key of this method is to restructure spectral feature images and choose convolution filters with a reasonable size, so that the spectral features of different land coverings in high dimensions can be extracted properly. In our experiments, proposed method was applied for hyperspectral data in several different situations, and preferable classification performance were obtained through relative parameters adjustment, which were given recommended scope during our comparative experiments.

Discriminant features captured from hyperspectral images can be used to accurately distinguish on-ground objects and materials for Earth observation. Typically, this process is hampered by inherent channel correlations and sample arrangements in the spectral and spatial domains. Deep learning networks play substantial roles in capturing discriminant features. However, their performances would be degraded when all information is modeled with equal importance. To tackle the aforementioned problem, we propose a deep spectral-spatial feature fusion based multi-scale adaptable attention network (SF ² MSA ² N). Our motivation is to strengthen the local considerable segments of spectral-spatial features by dual multi-scale adaptable strategies. We aim to promote adequate fusion and collaboration between spectral and spatial features. Dual multi-scale adaptable attention mechanisms are proposed to enhance the extraction ability for key informative features. Specifically, the spatial-wise attention provides an adaptable weight construction strategy for neighboring regions by unifying two metrics. The spectral-wise attention adaptively strengthens the significant spectral channels in local channel segments. Their adaptive multi-scale frameworks reduce the effect of imbalanced information. Inspired by the multimodal compact bilinear pooling (MCBP), we design an outer product based feature fusion strategy and its convolution based variant for better feature extraction performance. Multiple experiments conducted on four typical hyperspectral datasets prove that SF ² MSA ² N is capable to extract better features than several state-of-the-art techniques.

Hyperspectral image processing is faced with difficulties considering its redundant features and complex information. Studies on hyperspectral feature extraction in the deep learning domain have become increasingly popular. The mainstream techniques fully consider the spatial information in local neighborhoods when extracting spectral features by constructing deep neural networks. Deep generative models simulate the intrinsic structure of samples by adequately training, showing their potential values for signal processing. In this article, a convolutional two-stream network (cs²GAN-FE) based on the improved Wasserstein generative adversarial network (WGAN) is proposed for unsupervised hyperspectral spatial-spectral feature extraction. The improved WGAN is composed of one generator and one discriminator; the former perceives real data distributions, and the latter determines the attribution of generated data. The designed two-stream strategy is not a simple extension of a one-stream strategy and considers both the static spectral-spatial information and the dynamic spectral reflectance variation in multiple bands. Intrinsic spatial-spectral features are extracted by the trained discriminator considering sample distributions and feature relationships. The loss function is also improved for the unique structure of cs²GAN-FE. Various state-of-the-art techniques are chosen for comparison. Experimental results show the feasibility and potential of this network. Besides, experiments with the random split and the disjointed split both show that the proposed method can outperform other comparison techniques.

Remote sensing (RS) modalities from multi-sensor platforms, including hyperspectral (HS) and light detection and ranging (LiDAR) data, have been garnering increasing attention in overcoming the lack of information diversity. However, the specific modality correlation is always weakened and neglected. By reducing the spectral uncertainty, hyperspectral intrinsic image decomposition (HIID) has been proven to be effective in enhancing the HS image quality. It is an ill-posed problem and is challenged to capture the inherent modality connection. This paper proposes a shadow mask driven multimodal intrinsic image decomposition (smMIID) for HS and LiDAR data fusion. Notably, smMIID creatively clarifies and activates the potential of the crossmodal information in RS modalities and builds on the strength of the LiDAR elevation information when constructing HIID constraints. Our motivation is to overcome the deficiency of information diversity and modality correlation in existing IID based frameworks for better data fusion performance. Classic IID methods decompose the HS image into one reflectance component (RC) and one shading component (SC). There are three constraints in smMIID: 1) the fundamental constraint on RC and SC, 2) the HS-LiDAR hybrid gradient based constraint on RC and 3) the LiDAR gradient based constraint on SC. A shadow mask is obtained and utilized to guarantee spectral consistency when combining neighboring samples. Experiments on HS and LiDAR datasets prove that smMIID outperforms other techniques in terms of visualization and classification. The analyses are provided theoretically and experimentally to demonstrate that smMIID is robust to shadow mask generation.