3D depth cameras have become more and more popular in recent years. However, depth maps captured by these cameras can hardly be used in 3D reconstruction directly because they often suffer from low resolution and blurring depth discontinuities. Super resolution of depth maps is necessary. In depth maps, the edge areas play more important role and demonstrate distinct geometry directions compared with natural images. However, most existing super-resolution methods ignore this fact, and they can not handle depth edges properly. Motivated by this, we propose a compound method that combines multi-direction dictionary sparse representation and autoregressive (AR) models, so that the depth edges are presented precisely at different levels. In the patch level, the depth edge patches with geometry directions are well represented by the pre-trained multi-directional dictionaries. Compared with a universal dictionary, multiple dictionaries trained from different directional patches can represent the directional depth patch much better. In the finer pixel level, we utilize an adaptive AR model to represent the local correlation patterns in small areas. Extensive experimental results on both synthetic and real datasets demonstrate that, the proposed model outperforms state-of-the-art depth map super-resolution methods in terms of both quantitative metrics and subjective visual quality.

The edge information plays a key role in the restoration of a depth map. Most conventional methods assume that the color image and depth map are consistent in edge areas. However, complex texture regions in the color image do not match exactly with edges in the depth map. In this paper, firstly, we point out that in most cases the consistency between normal map and depth map is much higher than that between RGB-D pairs. Then we propose a dual normal-depth regularization term to guide the restoration of depth map, which constrains the edge consistency between normal map and depth map back and forth. Moreover, considering the bimodal characteristic of weight distribution that exists in depth discontinuous areas, a reweighted graph Laplacian regularizer is proposed to promote this bimodal characteristic. And this regularization is incorporated into a unified optimization framework to effectively protect the piece-wise smoothness(PWS) characteristics of depth map. By treating depth image as graph signal, the weight between two nodes is adapted according to its content. The proposed method is tested for both noise-free and noisy cases, and is compared against the state-of-the-art methods on both synthesis and real captured datasets. Extensive experimental results demonstrate the superior performance of our method compared with most state-of-the-art works in terms of both objective and subjective quality evaluations. Specifically, our method is more effective on edge areas and more robust to noises.

In the Coded Aperture Snapshot Spectral Imaging (CASSI) systems, hyperspectral images (HSIs) reconstruction methods are employed to recover 3D signals from 2D compressive measurements. Among these methods, deep unfolding networks exhibit the benefits of interpretability and high efficiency, but they still have some notable shortcomings. Firstly, existing methods primarily exploit the spatial-spectral domain information of HSIs, neglecting exploration of the frequency domain, which is also beneficial to 3D HSIs. Secondly, current unfolding networks have limited utilization of information between different stages, failing to fully explore their relevance and thereby limiting the effectiveness of the overall framework. To address these issues, in this paper, we propose an integrated framework with dual-domain feature fusion and multi-level memory enhancement. Specifically, the former represents the first attempt to utilize frequency domain information in the feature space of HSIs overcoming the limitation of spatial-spectral domain features and thereby improving the data expression ability of the network by extracting dual-domain features. Simultaneously, our verification experiments also show that the proposed dual-domain feature representation can indeed extract complementary feature information in HSIs. Moreover, the latter aims to use the structural characteristics of the U-Net network to fully extract the correlation of information between different stages by designing a multi-level memory enhancement network. Extensive experimental results on various datasets validate the superiority of the proposed approach in both subjective and objective outcomes. Our proposed method achieves an average of 0.4dB improvement over the best counterpart method. And the code can be obtained from the link: https://github.com/yingyangke/DFFMM.