Liquid crystal tunable filters (LCTF) are extensively used in hyperspectral imaging systems to successively acquire different spectral components of scenes by adjusting the center wavelength of the filter. However, the spectral and spatial resolutions of the imager are limited by the bandwidth of LCTF, and the pitch dimension of the detector, respectively. This paper applies compressive sensing principles to improve both of the spatial and spectral resolutions of the LCTF-based hyperspectral imaging system. An accurate transmission model of the LCTF is used to represent its bandpass filtering effects on the spectra. In addition, a random coded aperture placed behind the LCTF is used to modulate the spectral images in the spatial domain. Then, the three-dimensional encoded spectral images are projected onto a two-dimensional detector. Benefiting from the spectral-dependent transmission property of the LCTF, information of the entire spectrum is collected by a few snapshots using different center wavelengths of the LCTF. Super-resolution hyperspectral images can be reconstructed from a small set of compressive measurements by solving a convex optimization problem. Simulations and experiments show that the proposed method can effectively improve the spectral and spatial resolutions of traditional LCTF-based spectral imager without changing the structures of the LCTF and detector. Finally, a multi-channel spectral coding method is proposed to further increase the compression capacity of the system.
Coded apertures with random patterns are extensively used in compressive spectral imagers to sample the incident scene in the image plane. Random samplings, however, are inadequate to capture the structural characteristics of the underlying signal due to the sparsity and structure nature of sensing matrices in spectral imagers. This paper proposes a new approach for super-resolution compressive spectral imaging via adaptive coding. In this method, coded apertures are optimally designed based on a two-tone adaptive compressive sensing (CS) framework to improve the reconstruction resolution and accuracy of the hyperspectral imager. A liquid crystal tunable filter (LCTF) is used to scan the incident scene in the spectral domain to successively select different spectral channels. The output of the LCTF is modulated by the adaptive coded aperture patterns and then projected onto a low-resolution detector array. The coded aperture patterns are implemented by a digital micromirror device (DMD) with higher resolution than that of the detector. Due to the strong correlation across the spectra, the recovered images from previous spectral channels can be used as a priori information to design the adaptive coded apertures for sensing subsequent spectral channels. In particular, the coded apertures are constructed from the a priori spectral images via a two-tone hard thresholding operation that respectively extracts the structural characteristics of bright and dark regions in the underlying scenes. Super-resolution image reconstruction within a spectral channel can be recovered from a few snapshots of low-resolution measurements. Since no additional side information of the spectral scene is needed, the proposed method does not increase the system complexity. Based on the mutual-coherence criterion, the proposed adaptive CS framework is proved theoretically to promote the sensing efficiency of the spectral images. Simulations and experiments are provided to demonstrate and assess the proposed adaptive coding method. Finally, the underlying concepts are extended to a multi-channel method to compress the hyperspectral data cube in the spatial and spectral domains simultaneously.
Image segmentation is extensively used in remote sensing spectral image processing. Most of the existing region merging methods assess the heterogeneity or homogeneity using global or pre-defined parameters, which lack the flexibility to further improve the goodness-of-fit. Recently, the local spectral angle (SA) threshold was used to produce promising segmentation results. However, this method falls short of considering the inherent relationship between adjacent segments. In order to overcome this limitation, an adaptive SA thresholds methods, which combines the inter-segment and boundary homogeneities of adjacent segment pairs by their respective weights to refine predetermined SA threshold, is employed in a hybrid segmentation framework to enhance the image segmentation accuracy. The proposed method can effectively improve the segmentation accuracy with different kinds of reference objects compared to the conventional segmentation approaches based on the global SA and local SA thresholds. The results of the visual comparison also reveal that our method can match more accurately with reference polygons of varied sizes and types.
Polarized hyperspectral images can reflect the rich physicochemical characteristics of targets. Meanwhile, the contained plentiful information also brings great challenges to signal processing. Although compressive sensing theory provides a good idea for image processing, the simplified compression imaging system has difficulty in reconstructing full polarization information. Focused on this problem, we propose a two-step reconstruction method to handle polarization characteristics of different scales progressively. This paper uses a quarter-wave plate and a liquid crystal tunable filter to achieve full polarization compression and hyperspectral imaging. According to their numerical features, the Stokes parameters and their modulation coefficients are simultaneously scaled. The first Stokes parameter is reconstructed in the first step based on compressive sensing. Then, the last three Stokes parameters with similar order of magnitude are reconstructed in the second step based on previous results. The simulation results show that the two-step reconstruction method improves the reconstruction accuracy by 7.6 dB for the parameters that failed to be reconstructed by the non-optimized method, and reduces the reconstruction time by 8.25 h without losing the high accuracy obtained by the current optimization method. This feature scaling method provides a reference for the fast and high-quality reconstruction of physical quantities with obvious numerical differences.
Compressed sensing (CS) has been widely used in hyperspectral (HS) imaging to obtain hyperspectral data at a sub-Nyquist sampling rate, lifting the efficiency of data acquisition. Yet, reconstructing the acquired HS data via iterative algorithms is time consuming, which hinders the real-time application of compressed HS imaging. To alleviate this problem, this paper makes the first attempt to adopt convolutional neural networks (CNNs) to reconstruct three-dimensional compressed HS data by backtracking the entire imaging process, leading to a simple yet effective network, dubbed the backtracking reconstruction network (BTR-Net). Concretely, we leverage the divide-and-conquer method to divide the imaging process based on coded aperture tunable filter (CATF) spectral imager into steps, and build a subnetwork for each step to specialize in its reverse process. Consequently, BTR-Net introduces multiple built-in networks which performs spatial initialization, spatial enhancement, spectral initialization and spatial–spectral enhancement in an independent and sequential manner. Extensive experiments show that BTR-Net can reconstruct compressed HS data quickly and accurately, which outperforms leading iterative algorithms both quantitatively and visually, while having superior resistance to noise.
Adaptive coding is a recently emerging strategy for designing coded apertures using a-priori information of target layout to improve the reconstruction accuracy of compressed information. Based on the compressive sensing imaging system, this paper proposes a method for designing adaptive coded apertures by using the low-resolution a-prior information of every spectral band of the scene. This method also enables coded apertures containing both positive and negative coding information can be implemented in a compressive spectral imaging system that uses DMD or similar device as a coding device. Compared with other designs that utilized various kinds of a-prior information, this solution does not require additional equipment and is in real-time. The simulation results show the proposed real-time adaptive coded apertures method can attain better reconstruction performance than that of a random coding method at relative high compression ratio.