Geometric surface information such as depth maps and surface normals can be acquired by various methods such as stereo light fields, shape from shading and photometric stereo techniques. We compare several algorithms which deal with the combination of depth with surface normal information in order to reconstruct a refined depth map. The reasons for performance differences are examined from the perspective of alternative formulations of surface normals for depth reconstruction. We review and analyze methods in a systematic way. Based on our findings, we introduce a new generalized fusion method, which is formulated as a least squares problem and outperforms previous methods in the depth error domain by introducing a novel normal weighting that performs closer to the geodesic distance measure. Furthermore, a novel method is introduced based on Total Generalized Variation (TGV) which further outperforms previous approaches in terms of the geodesic normal distance error and maintains comparable quality in the depth error domain.
We present a system to extract surface orientation and albedos from a single shot image using three differently colored illumination sources. Photometric stereo allows one to extract local surface information such as normals or gradients. Traditionally, the local orientations and albedos are computed using serveral acquisitions of the same viewing angle and under varying illumination directions. In applications with moving objects, where the acquisition-as well as processing speed are essential, such setups are poorly suited. We propose a single shot decomposition using three differently colored light sources under defined illumination directions. To allow for a fast and regularized inference, we built a split U-shaped convolutional neural network, which takes a single shot input and estimates both the surface orientation and albedo simultaneously.
Multi-line scan systems have been introduced as linear light field cameras and subsequently for 3D ranging for industrial inline applications. Up to now there have been no viable calibration methods to determine intrinsic and extrinsic parameters of such a system which would allow (i) metric measurements and (ii) line-scan image geometric rectification. Our work closes the gap by exploiting special properties of a typical multi-line scan setup, which internally uses a fast area-scan sensor that can also be operated in the line-scan mode. This allows the use of standard calibration approaches to determine the intrinsic camera parameters. We introduce a novel method to compute extrinsic camera parameters w.r.t. the transport direction. Consecutively, the images are rectified for all constructed line-scan views. This takes into account estimated camera model parameters in order to generate an EPI-corrected linear light field that is suitable for accurate 3D reconstructions. Furthermore, we introduce a novel calibration target that is characteristic by an asymmetric central element as well as a tailored fast detection algorithm. The proposed method significantly improves the 3D reconstruction quality and allows for absolute 3D measurements in metric units using the multi-line scan setup. The performance of the proposed method is demonstrated on several representative real world examples.
We present a hybrid multi-line scan approach which enables simultaneous acquisition of light field & photometric stereo data. While light fields capture mostly large-scale surface deviations and rely on visible surface structures, photometric stereo is primarily sensitive to fine surface deviations and does not rely on visible structures. The combination of both approaches yields a solid performance for a large variety of depths, ranging from macro- to microscopic scales. Contrary to traditional photometric stereo, that relies on a strobed illumination, our approach uses two constant light sources which, however, generate multiple illumination geometries in different portions of the camera's field of view. Our object is moving on a conveyor belt during the acquisition process. Due to our multi-line scan sensor the object is observed from several viewing angles. The object's movement is causing each object point to be illuminated under several illumination directions. Hence, during our acquisition process the object points are captured under all feasible viewing angles and lighting conditions. In our system, surface normals are derived making use of the Lambert's cosine law. However, due to the lack of illuminations spanning orthogonally to the transport direction, the surface normals can be inferred only in the transport direction. We present a variational approach for 3D depth reconstruction designed specifically for our hybrid setup that jointly takes into account the light field as well as photometric stereo depth cues and provides one globally consistent solution. Depth maps obtained by the proposed algorithm show both the large-scale accuracy as well as sensitivity to fine surface details.
The acquisition of partial BRDF measurements using light field cameras and several illumination directions raises critical questions regarding the accuracy of inferences based on that data. Therefore, we attempt to verify the quality of the reconstruction of a full BRDF using partial input data. A dataset that provides a densely sampled BRDF was used, both in viewing and illumination directions. We show the reconstruction of dense BRDFs when the viewing angles are limited to top central regions, while the illumination angles are not reduced and are positioned in the shape of a half sphere around the material object, these properties are characteristic of data provided by plenoptic cameras paired with a photometric light dome. The partial reconstruction of the dense BRDF out of data is achieved by utilizing convolutional neural networks. We obtain a competitive full reconstruction when up to 2/3 of the BRDF is unknown.
This work focuses on the compensation of motion artefacts that may occur during a line scan acquisition and can be detected with our multi-line scan imaging system [10]. These artefacts are caused by fluctuations of the transport velocity that are not correctly reflected by the camera trigger, and are especially visible at high magnifications. We reduce such artefacts by analyzing the light field acquired with our system. Specifically, we use a variational formulation to design a warping function, such that lines that are acquired too early or too late are stretched or squeezed appropriately. To this end, we exploit the information comprised in the light field, i.e., control the estimation of the warping function by comparing light field views and enforce uniform spacing between line acquisitions. The proposed approach enables our system to perform the multi-line scan light field imaging at virtually any magnification independent from the transport and trigger quality. We demonstrate the capabilities of our approach for various objects by comparing 3D reconstructions from unprocessed acquisitions and our corrected acquisitions. Our approach significantly reduces artefacts in light fields and in 3D reconstructions that are generated from them.