Occlusions will reduce the performance of systems in many computer vision applications with discontinuous surfaces of 3D scenes. We explore a signal-processing framework of occlusions based on the light ray visibility to improve the rendering quality of views. An occlusion field (OCF) theory is derived by calculating the relationship between the occluded light rays and the nonoccluded light rays to quantify the occlusion degree (OCD). The OCF framework can describe the various in-scene information captured by the changes in the camera configuration (i.e., position and direction) through a quantitative description of the occlusion information. From a spectral analysis of the OCF, we mathematically derive analytical functions to determine the changing relationship between the scene and the camera configuration. A reconstruction filter can be designed to achieve interference cancellation and compensate for the missing information caused by the occlusions. Our measurements of different occlusions using this OCF framework included both synthetic and actual scenes. The experimental results show that the proposed OCF framework can improves the rendering quality of views and outperforms other known occlusion quantization schemes in a complex scene.

Occlusions are common phenomena in light field rendering (LFR) technology applications. The 3-D spatial structures of some features may be missing or incorrect when capturing some samples due to occlusion discontinuities. Most prior works on LFR, however, have neglected occlusions from other objects in 3-D scenes that do not participate in the capturing and rendering of the light field. To improve rendering quality, this report proposes an occlusion probability learning framework (OPLF) based on a deep Boltzmann machine (DBM) to compensate for the occluded information. In the OPLF, an occlusion probability density model is applied to calculate the visibility scores, which are modeled as hidden variables. Additionally, the probability of occlusion is related to the visibility, the camera configuration (i.e., position and direction), and the relationship between the occlusion object and occluded object. Furthermore, a deep probability model based on the OPLF is used for learning the occlusion relationship between the camera and object in multiple layers. The proposed OPLF can optimize the LFR quality. Finally, to verify the claimed performance, we also compare the OPLF with the most advanced occlusion theory and light field reconstruction algorithms. The experimental results show that the proposed OPLF outperforms other known occlusion quantization schemes.

We present a signal-processing framework for image-assisted geometry measurement in the image-based rendering (IBR). We study the utilized geometry information and estimating minimum sampling rate of the IBR. Our method combines decomposing a complex scene geometry into a collection of simpler structures on a block-by-block basis. The automatic simpler structure selection can be interactively refined by detected single salient points. In this manner, we reduce the spectral analysis problem of an irregular object to that of a simpler structure. Predictions on the frequency content can then be used to control the sampling rate. This extends previous work in which the IBR sampling is analyzed and estimated for nonuniform sampling. Extensive experimental evaluation demonstrates that our geometry simplification method significantly outperforms competing algorithms. Additionally, the minimum sampling rate of the IBR necessary for alias-free rendering will be reduced as the number of simpler structures increases.

In this report, we studied the sampling synchronization of a discrete signal in the receiver of a communication system and found that the frequency of the received signal usually exhibits some unpredictable deviations. We observed many harmonics caused by the frequency deviations of the discrete received signal. These findings indicate that signal sampling synchronization is an important research technique when using discrete Fourier transforms (DFT) to analyze the harmonics of discrete signals. We investigated the influence of these harmonics on the performance of signal sampling and studied the frequency estimation of the received signal. Based on the frequency estimation of the received signal, the sampling rate of the discrete signal was converted using a modified Farrow filter to achieve sampling synchronization for the received signal. The algorithm discussed here can be applied to sampling synchronization for monitoring and control systems. Finally, simulations and experimental results are presented.

We present a sparse reconstruction light field (SRLF) framework using compressed sensing theory. Light field signals have sparsity in some specific phenomena, such as nonocclusions, smooth surfaces and texture uniformity. We study the light field sparsity to analyze the sparse basis and measurement matrix of the light field in a complicated scene. This extends previous work on light field sampling that considered either spatial or angular dimensions, which can be used to control the sampling rate of the light field. Furthermore, the sparse Bayes learning (SBL) algorithm is applied to the reconstruction of sparsely sampled light fields. We derive a learning machine for the light field SBL algorithm, which can improve the rendering quality based on a given set of captured multiview images. The proposed SRLF compares favorably with state-of-the-art light field sampling and reconstruction techniques. The innovation of the SRLF is to use compressed sensing theory to further reduce the light field sampling rate. We conduct a detailed derivation of the mathematical theory of light field sparseness.

Light field rendering is an effective multi-view reconstruction technology. The current multi-view reconstruction methods still has the problem of low view synthesis quality in complex scenes, such as occlusion and non Lambertian surfaces. In this article, we consider undistorted multiview reconstruction and develop a phase retrieval method for light field (PRLF) signals to improve the rendering quality. Our approach utilizes the phase spectrum in both the time domain and frequency domain of the discrete light field because the phase of the light field carries more scene information. The phase corrections are “light field truncation” in the time domain and “phase replacement” in the frequency domain. The iterative algorithm used provides high-quality estimated results for the relative phase of the light field. By the estimated phase results, we derive a multiview reconstruction method to improve the rendering quality. Finally, to verify the claimed performance, we also compare the PRLF with the most advanced light field reconstruction algorithms. The experimental results show that the proposed PRLF outperforms other known reconstruction schemes.