Transformers have intrigued the vision research community with their state-of-the-art performance in natural language processing. With their superior performance, transformers have found their way into the field of hyperspectral image classification and achieved promising results. In this article, we harness the power of transformers to conquer the task of hyperspectral unmixing and propose a novel deep neural network-based unmixing model with transformers. A transformer network captures nonlocal feature dependencies by interactions between image patches, which are not employed in convolutional neural network (CNN) models, and hereby has the ability to enhance the quality of the endmember spectra and the abundance maps. The proposed model is a combination of a convolutional autoencoder and a transformer. The hyperspectral data is encoded by the convolutional encoder. The transformer captures long-range dependencies between the representations derived from the encoder. The data are reconstructed using a convolutional decoder. We applied the proposed unmixing model to three widely used unmixing datasets, that is, Samson, Apex, and Washington DC Mall, and compared it with the state-of-the-art in terms of root mean squared error and spectral angle distance. The source code for the proposed model will be made publicly available at https://github.com/preetam22n/DeepTrans-HSU .

In this article, we introduce a deep learning-based technique for the linear hyperspectral unmixing problem. The proposed method contains two main steps. First, the endmembers are extracted using a geometric endmember extraction method, i.e., a simplex volume maximization in the subspace of the data set. Then, the abundances are estimated using a deep image prior. The main motivation of this work is to boost the abundance estimation and make the unmixing problem robust to noise. The proposed deep image prior uses a convolutional neural network to estimate the fractional abundances, relying on the extracted endmembers and the observed hyperspectral data set. The proposed method is evaluated on simulated and three real remote sensing data for a range of SNR values (i.e., from 20 to 50 dB). The results show considerable improvements compared to state-of-the-art methods. The proposed method was implemented in Python (3.8) using PyTorch as the platform for the deep network and is available online: https://github.com/BehnoodRasti/UnDIP .

In this article, we propose a minimum simplex convolutional network (MiSiCNet) for deep hyperspectral unmixing. Unlike all the deep learning-based unmixing methods proposed in the literature, the proposed convolutional encoder–decoder architecture incorporates spatial information and geometrical information of the hyperspectral data in addition to the spectral information. The spatial information is incorporated using convolutional filters and implicitly applying a prior on the abundances. The geometrical information is exploited by incorporating a minimum simplex volume penalty term in the loss function for the endmember estimation. This term is beneficial when there are no pure material pixels in the data, which is often the case in real-world applications. We generated simulated datasets, where we consider two different no-pure pixel scenarios. In the first scenario, there are no pure pixels but at least two pixels on each facet of the data simplex (i.e., mixtures of two pure materials). The second scenario is a complex case with no pure pixels and only one pixel on each facet of the data simplex. In addition, we evaluate the performance of MiSiCNet in three real datasets. The experimental results confirm the robustness of the proposed method to both noise and the absence of pure pixels. In addition, MiSiCNet considerably outperforms the state-of-the-art unmixing approaches. The results are given in terms of spectral angle distance in degree for the endmember estimation and the root mean square error in percentage for the abundance estimation. MiSiCNet was implemented in Python (3.8) using PyTorch as the platform for the deep network and is available online: https://github.com/BehnoodRasti/MiSiCNet .

In this letter, we propose a sparse unmixing technique using a convolutional neural network (SUnCNN) for hyperspectral images. SUnCNN is the first deep learning-based technique proposed for sparse unmixing. It uses a deep convolutional encoder–decoder to generate the abundances relying on a spectral library. We reformulate the sparse unmixing into an optimization over the deep network’s parameters. Therefore, the deep network learns in an unsupervised manner to map a fixed input into the sparse optimum abundances. Additionally, SUnCNN holds the sum-to-one constraint using a softmax activation layer. The proposed method is compared with the state-of-the-art using two synthetic datasets and one real hyperspectral dataset. The overall results confirm that the proposed method outperforms the other ones in terms of signal to reconstruction error (SRE). Additionally, SUnCNN shows visual superiority for both real and synthetic datasets compared with the competing techniques. The proposed method was implemented in Python (3.8) using PyTorch as the platform for the deep network and is available online: https://github.com/BehnoodRasti/SUnCNN .

This article proposes a blind nonlinear unmixing technique for intimate mixtures using the Hapke model and convolutional neural networks (HapkeCNN). We use the Hapke model and a fully convolutional encoder–decoder deep network for the nonlinear unmixing. Additionally, we propose a novel loss function that includes three terms; 1) a quadratic term based on the Hapke model, that captures the nonlinearity; 2) the reconstruction error of the reflectances, to ensure the fidelity of the reconstructed reflectance; and 3) a minimum volume total variation (TV) term that exploits the geometrical information to estimate the endmembers in the absence of pure pixels in the hyperspectral data. The proposed method is evaluated using two simulated and two real datasets. We compare the results of endmember and abundance estimation with a number of nonlinear, and projection-based linear unmixing techniques. The experimental results confirm that HapkeCNN considerably outperforms the state-of-the-art nonlinear approaches. The proposed method was implemented in Python (3.9) using PyTorch as the platform for the deep network and is available at: https://github.com/BehnoodRasti/HapkeCNN .

Hyperspectral unmixing is a crucial technique in remote sensing data processing that aims to estimate component information from mixed pixels in hyperspectral images. Most existing deep learning-based hyperspectral unmixing models employ autoencoder networks to reconstruct hyperspectral images and estimate abundance maps. Here, the weight between the reconstructed and the Softmax layers is used to extract/estimate endmember signatures. However, autoencoders are heavily dependent on initial weights, which introduces inherent randomness, potentially compromising unmixing accuracy. To address this issue, in this paper we present a new dual-feature fusion network (DFFN) for enhanced hyperspectral unmixing. Our DFFN mainly consists of four modules: 1) a feature fusion module (FFM), 2) an abundance estimation module (AEM), 3) an endmember estimation module (EEM), and 4) a reconstruction module (RE). Firstly, FFM calculates spectral and spatial similarities and then enhances the hyperspectral image by matrix multiplications with similarity matrices. Second, AEM takes the enhanced hyperspectral image as input and uses convolutional layers to estimate abundances and reconstruct the image. Next, the reconstructed image is fed into EEM to automatically estimate endmembers. RE performs the final reconstruction through matrix multiplication of the estimated endmembers and abundances. Experiments on synthetic and real hyperspectral datasets, together with a comparison with state-of-the-art techniques, demonstrate the superiority of our newly proposed DFFN. The full code is released at https://github.com/xuanwentao for public evaluation.