Hyperspectral (HS) images are characterized by approximately contiguous spectral information, enabling the fine identification of materials by capturing subtle spectral discrepancies. Due to their excellent locally contextual modeling ability, convolutional neural networks (CNNs) have been proven to be a powerful feature extractor in HS image classification. However, CNNs fail to mine and represent the sequence attributes of spectral signatures well due to the limitations of their inherent network backbone. To solve this issue, we rethink HS image classification from a sequential perspective with transformers and propose a novel backbone network called SpectralFormer . Beyond bandwise representations in classic transformers, SpectralFormer is capable of learning spectrally local sequence information from neighboring bands of HS images, yielding groupwise spectral embeddings. More significantly, to reduce the possibility of losing valuable information in the layerwise propagation process, we devise a cross-layer skip connection to convey memory-like components from shallow to deep layers by adaptively learning to fuse “soft” residuals across layers. It is worth noting that the proposed SpectralFormer is a highly flexible backbone network, which can be applicable to both pixelwise and patchwise inputs. We evaluate the classification performance of the proposed SpectralFormer on three HS datasets by conducting extensive experiments, showing the superiority over classic transformers and achieving a significant improvement in comparison with state-of-the-art backbone networks. The codes of this work will be available at https://github.com/danfenghong/IEEE_TGRS_SpectralFormer for the sake of reproducibility.

Depending on a large-scale paired dataset of low-resolution hyperspectral images (LrHSI), high-resolution multispectral images (HrMSI), and corresponding high-resolution hyperspectral images (HrHSI), the supervised paradigm has achieved impressive performance in the hyperspectral image super-resolution (HISR). However, the intrinsic data-intensive manner hinders its further application in real scenarios. Fortunately, deep image prior (DIP) allows us to achieve unsupervised SR by solely utilizing degraded observations. However, its potential to accurately model complicated hyperspectral priors is still not fully exploited owing to the following two factors: i) Existing methods tend to reconstruct the unknown HrHSI directly from a randomly generated noise, leaving it hard to leverage the scene-relevant information for prior learning; ii) The vanilla architecture is handcrafted for the generator network, which shows limitations in feature representation and hence fails to characterize the complicated image properties. To unleash the potential of DIP for the HISR task, we propose an enhanced DIP network, called EDIP-Net, by addressing the aforementioned impediments. Specifically, EDIP-Net is built with a two-stage four-component scheme, with a zero-shot learning (ZSL) stage for input image establishment and a deep image generation stage for prior learning. First, we exploit the cross-scale spectral relationship inside the observations and hence design a degradation learning network to generate paired training samples from the observations themselves. As such, two image-coarse estimations are derived in a ZSL manner by learning an interactive spectral learning network. By replacing random noise with two estimations, we design a double U-shape architecture for the generator network to capture their hyperspectral prior, each independently generating one HrHSI candidate. Under this premise, we further propose a degradation-aware decision fusion strategy to integrate the optimal results in a pixel-to-pixel manner. Extensive experiments demonstrate our superiority in achieving high-quality SR performance. The code will be available at https://github.com/JiaxinLiCAS.

In recent years, deep learning (DL) has attracted increasing attention in hyperspectral unmixing (HU) applications due to its powerful learning and data fitting ability. The autoencoder (AE) framework, as an unmixing baseline network, achieves good performance in HU by automatically learning low-dimensional embeddings and reconstructing data. Nevertheless, the conventional AE-based architecture, which focuses more on the pixel-level reconstruction loss, tends to lose some significant detailed information of certain materials (e.g., material-related properties) in the reconstruction process. Therefore, inspired by the perception mechanism, we propose a cycle-consistency unmixing network, called CyCU-Net, by learning two cascaded AEs in an end-to-end fashion, to enhance the unmixing performance more effectively. CyCU-Net is capable of reducing the detailed and material-related information loss in the process of reconstruction by relaxing the original pixel-level reconstruction assumption to cycle consistency dominated by the cascaded AEs. More specifically, cycle consistency can be achieved by a newly proposed self-perception loss, which consists of two spectral reconstruction terms and one abundance reconstruction term. By taking advantage of the self-perception loss in the network, the high-level semantic information can be well preserved in the unmixing process. Moreover, we investigate the performance gain of CyCU-Net with extensive ablation studies. Experimental results on one synthetic and three real hyperspectral data sets demonstrate the effectiveness and competitiveness of the proposed CyCU-Net in comparison with several state-of-the-art unmixing algorithms.

Owing to the rapid development of sensor technology, hyperspectral (HS) remote sensing (RS) imaging has provided a significant amount of spatial and spectral information for the observation and analysis of Earth’s surface at a distance of data acquisition devices. The recent advancement and even revolution of HS RS techniques offer opportunities to realize the potential of various applications while confronting new challenges for efficiently processing and analyzing the enormous HS acquisition data. Due to the maintenance of the 3D HS inherent structure, tensor decomposition has aroused widespread concern and spurred research in HS data processing tasks over the past decades. In this article, we aim to present a comprehensive overview of tensor decomposition, specifically contextualizing the five broad topics in HS data processing: HS restoration, compressive sensing (CS), anomaly detection (AD), HS–multispectral (MS) fusion, and spectral unmixing (SU). For each topic, we elaborate on the remarkable achievements of tensor decomposition models for HS RS, with a pivotal description of the existing methodologies and a representative exhibition of experimental results. As a result, the remaining challenges of the follow-up research directions are outlined and discussed from the perspective of actual HS RS practices and tensor decomposition merged with advanced priors and even deep neural networks. This article summarizes different tensor decomposition-based HS data processing methods and categorizes them into different classes, from simple adoptions to complex combinations with other priors for algorithm beginners. We expect that this survey provides new investigations and development trends for experienced researchers to some extent.

In synthetic aperture radar automatic target recognition (SAR-ATR), target information is usually propagated and reserved in complex-valued form, namely, magnitude information and phase information. However, most of the existing SAR target recognition methods only focus on real-valued (magnitude information) calculations and ignore the phase information of targets, yielding poor recognition performance. To overcome this limitation, this article proposes a multi-stream (MS) feature fusion SAR target recognition method based on complex-valued operations, called MS complex-valued networks (MS-CVNets), to utilize the phase information of the target effectively. First of all, a series of complex-valued operation blocks are constructed to satisfy the network training in the complex field, such as complex convolution, complex batch normalization, complex activation, complex pooling, and complex full connection. Besides, an MS structure is employed by applying different convolution kernels to extract multiscale information of targets, further enhancing the representation ability of the model. Experimental results on the moving and stationary target acquisition and recognition (MSTAR) dataset illustrate that compared with the current state-of-the-art real-valued based models, MS-CVNets can achieve better recognition results under both standard operating conditions (SOCs) and extended operating conditions (EOCs), validating the effectiveness and superiority of the proposed method.

Deep learning (DL) has aroused wide attention in hyperspectral unmixing (HU) owing to its powerful feature representation ability. As a representative of unsupervised DL approaches, autoencoder (AE) has been proven to be effective to better capture nonlinear components of hyperspectral images than the traditional model-driven linearized methods. However, only using hyperspectral images for unmixing fails to distinguish objects in complex scene, especially for different endmembers with similar materials. To overcome this limitation, we propose a novel multimodal unmixing network for hyperspectral images, called MUNet, by considering the height differences of light detection and ranging (LiDAR) data in a squeeze-and-excitation (SE)-driven attention fashion to guide the unmixing process, yielding performance improvement. MUNet is capable of fusing multimodal information and using the attention map derived by LiDAR to aid network that focuses on more discriminative and meaningful spatial information regarding scenes. Moreover, attribute profile (AP) is adopted to extract the geometrical structures of different objects to better model the spatial information of LiDAR. Experimental results on synthetic and real datasets demonstrate the effectiveness and superiority of the proposed method compared with several state-of-the-art unmixing algorithms. The codes will be available at https://github.com/hanzhu97702/IEEE_TGRS_MUNet , contributing to the remote sensing community.