Hyperspectral imaging enables many versatile applications for its competence in capturing abundant spatial and spectral information, which is crucial for identifying substances. However, the devices for acquiring hyperspectral images are typically expensive and very complicated, hindering the promotion of their application in consumer electronics, such as daily food inspection and point-of-care medical screening, etc. Recently, many computational spectral imaging methods have been proposed by directly reconstructing the hyperspectral information from widely available RGB images. These reconstruction methods can exclude the usage of burdensome spectral camera hardware while keeping a high spectral resolution and imaging performance. We present a thorough investigation of more than 25 state-of-the-art spectral reconstruction methods which are categorized as prior-based and data-driven methods. Simulations on open-source datasets show that prior-based methods are more suitable for rare data situations, while data-driven methods can unleash the full potential of deep learning in big data cases. We have identified current challenges faced by those methods (e.g., loss function, spectral accuracy, data generalization) and summarized a few trends for future work. With the rapid expansion in datasets and the advent of more advanced neural networks, learnable methods with fine feature representation abilities are very promising. This comprehensive review can serve as a fruitful reference source for peer researchers, thus paving the way for the development of computational hyperspectral imaging.

Deep convolutional neural network based video super-resolution (SR) models have achieved significant progress in recent years. Existing deep video SR methods usually impose optical flow to wrap the neighboring frames for temporal alignment. However, accurate estimation of optical flow is quite difficult, which tends to produce artifacts in the super-resolved results. To address this problem, we propose a novel end-to-end deep convolutional network that dynamically generates the spatially adaptive filters for the alignment, which are constituted by the local spatio-temporal channels of each pixel. Our method avoids generating explicit motion compensation and utilizes spatio-temporal adaptive filters to achieve the operation of alignment, which effectively fuses the multi-frame information and improves the temporal consistency of the video. Capitalizing on the proposed adaptive filter, we develop a reconstruction network and take the aligned frames as input to restore the high-resolution frames. In addition, we employ residual modules embedded with channel attention as the basic unit to extract more informative features for video SR. Both quantitative and qualitative evaluation results on three public video datasets demonstrate that the proposed method performs favorably against state-of-the-art super-resolution methods in terms of clearness and texture details.

Optical systems have been crucial for versatile applications such as consumer electronics, remote sensing and biomedical imaging. Designing optical systems has been a highly professional work due to complicated aberration theories and intangible rules-of-thumb, hence neural networks are only coming into this realm until recent years. In this work, we propose and implement a generic, differentiable freeform raytracing module, suitable for off-axis, multiple-surface freeform/aspheric optical systems, paving the way toward a deep learning-based optical design method. The network is trained with minimal prior knowledge, and it can infer numerous optical systems after a one-time training. The presented work unlocks great potential for deep learning in various freeform/aspheric optical systems, and the trained network could serve as an effective, unified platform for generating, recording, and replicating good initial optical designs.

The commonly used atmospheric model in image dehazing cannot hold in real cases. Although deep end-to-end networks were presented to solve this problem by disregarding the physical model, the transmission map in the atmospheric model contains significant haze density information, which cannot simply be ignored. In this article, we propose a novel hierarchical density-aware dehazing network, which consists of a the densely connected pyramid encoder, a density generator, and a Laplacian pyramid decoder. The proposed network incorporates density estimation but alleviates the constraint of the atmospheric model. The predicted haze density then guides the Laplacian pyramid decoder to generate a haze-free image in a coarse-to-fine fashion. In addition, we introduce a multiscale discriminator to preserve global and local consistency for dehazing. We conduct extensive experiments on natural and synthetic hazy images, which prove that the proposed model performs favorably against the state-of-the-art dehazing approaches.

In this work, we present a multifield direct design method for ultrashort throw ratio projection optics. The multifield design method allows us to directly calculate two freeform mirror profiles, which are fitted by odd polynomials and imported into an optical design program as an excellent starting point. Afterward, these two mirrors are represented by XY polynomial freeform surfaces for further optimization. The final configuration consists of an off-the-shelf projection lens and two XY polynomial freeform mirrors to greatly shorten the regular projection distance from 2 m to 48 cm for a 78.3 inch diagonal screen. The values of the modulation transfer function for the optimized freeform mirror system are improved to over 0.6 at 0.5 lp/mm, in comparison with its rotationally symmetric counterpart’s 0.43, and the final distortion is less than 1.5%, showing a very good and well-tailored imaging performance over the entire field of view.

Recently, freeform optics has been widely used due to its unprecedented compactness and high performance, especially in the reflective designs for broad-wavelength imaging applications. Here, we present a generalized differentiable ray tracing approach suitable for most optical surfaces. The established automated freeform design framework simultaneously calculates multi-surface coefficients with merely the system geometry known, very fast for generating abundant feasible starting points. In addition, we provide a “double-pass surface” strategy with desired overlap (not mutually centered) that enables a component reduction for very compact yet high-performing designs. The effectiveness of the method is firstly demonstrated by designing a wide field-of-view, fast f-number, four-mirror freeform telescope. Another example shows a two-freeform, three-mirror, four-reflection design with high compactness and cost-friendly considerations with a double-pass spherical mirror. The present work provides a robust design scheme for reflective freeform imaging systems in general, and it unlocks a series of new ‘double-pass surface’ designs for very compact, high-performing freeform imaging systems.