Light field cameras can capture spatial and angular information of light with a single exposure, but they suffer from low spatial resolution which limits their performance in practical use. Super-resolution (SR) methods has been used to improve spatial resolution of light field images, but most of them do not take full advantages of the particular structure of light field. In this letter, we present a SR method to obtain light field images with high quality and geometric consistency via a combined deep convolutional neural networks (DCNN) framework. The spatial resolution of subaperture images are enhanced separately by a single image super-resolution (SISR) DCNN. Then an epipolar plane image (EPI) enhancement DCNN is proposed to restore the geometric consistency of these images. Experimental results show that our method achieves the state-of-the-art performance on both quantitative and qualitative evaluations.
Hyperspectral unmixing, which decomposes mixed pixels into the endmembers and corresponding abundances, is an important image process for the further application of hyperspectral images (HSIs). Lately, the unmixing problem has been solved using deep learning techniques, particularly autoencoders (AEs). However, the majority of them are based on the simple linear mixing model (LMM), which disregards the spectral variability of endmembers in different pixels. In this article, we present a multi-attention AE network (MAAENet) based on the extended LMM to address the issue of the spectral variability problem in real scenes. Moreover, the majority of AE networks ignore the global spatial information in HSIs and operate pixel- or patch-wise. We employ attention mechanisms to design a spatial–spectral attention (SSA) module that can deal with the band redundancy in HSIs and extract global spatial features through spectral correlation. Moreover, noticing that the mixed pixels are always present in the intersection of different materials, a novel sparse constraint based on spatial homogeneity is designed to constrain the abundance and abstract local spatial features. Ablation experiments are conducted to verify the effectiveness of the proposed AE structure, SSA module, and sparse constraint. The proposed method is compared with several state-of-the-art unmixing methods and exhibits competitiveness on both synthetic and real datasets.
Hyperspectral images (HSI) frequently have inadequate spatial resolution, which hinders numerous applications for the images. High resolution multispectral image (MSI) has been fused with HSI to reconstruct images with both high spatial and high spectral resolutions. In this paper, we propose a generative adversarial network (GAN)-based unsupervised HSI-MSI fusion network. In the generator, two coupled autoencoder nets decompose HSI and MSI into endmembers and abundances for fusing high resolution HSI through the linear mixing model. The two autoencoder nets are connected by a degradation-generation (DG) block, which further improves the accuracy of the reconstruction. Additionally, a coordinate multi-attention net (CMAN) is designed to extract more detailed features from the input. Driven by the joint loss function, the proposed method is straightforward and easy to execute in an end-to-end training manner. The experimental results demonstrate that the proposed strategy outperforms the state-of-art methods.
In single disperser coded aperture snapshot spectral imaging (SD-CASSI) systems, many methods have been developed to reconstruct hyperspectral images (HSIs) from compressed measurements. Among these, deep learning (DL)-based methods have stood out, relying on powerful DL networks. However, the solidified structure of DL-based methods limits their adaptability. Moreover, they are often based on a model that neglects the dispersion process and instead emphasizes the encoding-compression process. Furthermore, research on optimization-based methods designed especially for SD-CASSI is lacking. In this article, we propose a comprehensive two-step projection imaging model for SD-CASSI that includes both spectral shearing projection and encoding-compression projection. Based on this model, we derive a tensor-based optimization framework that incorporates with the nonlocal low-rank tensor (NLRT) prior. In particular, NLRT extracts inherent spatial structural information from the measurements and employs it to guide the clustering of spatial–spectral similar HSI blocks. A CANDECOMP/PARAFAC (CP) low-rank regularizer is introduced to constrain the low-rank property of HSI block clusters. After that, we develop a solution framework based on the alternating direction method of multiplier (ADMM) approach. Comprehensive experiments demonstrate that our NLRT method outperforms state-of-the-art methods in terms of flexibility and performance. The source code and data of this article are publicly available at https://github.com/sdnjyxr/NLRT .
8 November 2023 in IEEE Transactions on Instrumentation and Measurement
Spectra reflect the essential features of objects and have important applications in many fields, such as remote sensing, biology, food testing, etc. The multispectral light field imager (MSLFI) can simultaneously capture spatial-spectral datacube and be used for detecting and monitoring dynamic targets. However, the spectral aliasing and high-level noise of the system are the main factors that degrade its spectral image quality and limit its practical applications. In this paper, we propose an adaptive sparse representation (ASR) method to reconstruct the target spectrum of MSLFI with high noise levels. A rough result is reconstructed with a redundant dictionary at first. By recognizing peak features, the algorithm adaptively selects a sub-dictionary to conduct a secondary optimization. Besides, a position-constrained calibration method is introduced to obtain a denoised and sparse spectral aliasing matrix. The simulation and experimental results showed that the proposed ASR method can flexibly handle spectra with various curvatures and effectively improve the reconstruction accuracy of different spectra.