We present a novel algorithm called crosspatch-based rolling label expansion for accurate stereo matching. This optimization-based approach can effectively estimate the 3D label of each pixel from huge and infinite label space and then generate a continuous disparity map. The algorithm has two obvious characteristics when compared with the traditional label expansion algorithms. The first feature is the cross-based multilayer structure, where each layer contains a series of cross patches with adaptive shapes, reflecting the edge structure of objects on the image. Besides, such cross patches are non-overlapping and independent, satisfying the submodular property for employing graph cuts. The second feature is the rolling optimization, that firstly generates new label proposal by expanding candidate labels within cross patches, then globally updates labels for the whole image using a proposed rolling move. The experimental results show the high matching accuracy of our method, both in pixel level and subpixel level. According to the latest ranking list of Middlebury 3.0 benchmark, our method is one of the best stereo matching algorithms.

ICP is a well-known method for point cloud registration but it only uses geometric information to do this, which will result in bad results in some similar structures. Adding color information when registering will improve the performance. However, color information of point cloud, such as gray, varies differently under different lighting conditions. Using gray as the color information to register can cause large errors and even wrong results. To solve this problem, we propose a color point cloud registration algorithm based on hue, which has good robustness at different lighting conditions. We extract the hue component according to the color information of point clouds and make the hue distribution of the tangent plane continuous. The error function consists of color and geometric error of two point clouds under the current transformation. We optimize the error function using the Gauss–Newton method. If the value of the error function is less than the preset threshold or the maximum number of iterations is reached, the current transformation relationship is required. We use RGB-D Scenes V2 dataset to evaluate our algorithm and the results show that the average recall of our algorithms is 8.63% higher than that of some excellent algorithms, and its RMSE of 14.3% is lower than that of the other compared algorithms.

Patch-based image inpainting methods iteratively fill the missing region via searching the best sample patch from the source region. However, most of the existing approaches basically use the fixed size of patch regardless of content features nearby, which may lead to inpainting defects. Also, global match is needed for searching the best sample patch, but only to fill one target patch in each iteration, resulting in low efficiency. To handle the issues above, we first evaluate the nonuniformity in an image, by which the patch size is adaptively determined. Moreover, we divide the source region into multiple non-overlapping subregions with different nonuniformity levels, and the patch match proceeds in every subregion, respectively. This strategy not only saves the match time for single target patch, but also reduces the mismatch, and enables the simultaneous filling of multiple target patches in a single iteration. Experimental results show that in comparison to previous patch-based works, our method has achieved further improvement both in quality and efficiency. We believe our method could provide a new way for patch match with better accuracy and efficiency in image inpainting tasks.

Depth image-based rendering (DIBR) plays an important role in 3D video and free viewpoint video synthesis. However, artifacts might occur in the synthesized view due to viewpoint changes and stereo depth estimation errors. Holes are usually out-of-field regions and disocclusions, and filling them appropriately becomes a challenge. In this paper, a virtual view synthesis approach based on asymmetric bidirectional DIBR is proposed. A depth image preprocessing method is applied to detect and correct unreliable depth values around the foreground edges. For the primary view, all pixels are warped to the virtual view by the modified DIBR method. For the auxiliary view, only the selected regions are warped, which contain the contents that are not visible in the primary view. This approach reduces the computational cost and prevents irrelevant foreground pixels from being warped to the holes. During the merging process, a color correction approach is introduced to make the result appear more natural. In addition, a depth-guided inpainting method is proposed to handle the remaining holes in the merged image. Experimental results show that, compared with bidirectional DIBR, the proposed rendering method can reduce about 37% rendering time and achieve 97% hole reduction. In terms of visual quality and objective evaluation, our approach performs better than the previous methods.

The depth image based rendering (DIBR) is a popular technology for 3D video and free viewpoint video (FVV) synthesis, by which numerous virtual views can be generated from a single reference view and its depth image. However, some artifacts are produced in the DIBR process and reduce the visual quality of virtual view. Due to the diversity of artifacts, effectively handling them becomes a challenging task. In this paper, an artifact handling method based on depth image is proposed. The reference image and its depth image are extended to fill the holes that belong to the out-of-field regions. A depth image preprocessing method is applied to project the ghosts to their correct place. The 3D warping process is optimized by an adaptive one-to-four method to deal with the cracks and pixel overlapping. For disocclusions, we calculate depth and background terms of the filling priority based on depth information. The search for the best matching patch is performed simultaneously in the reference image and the virtual image. Moreover, adaptive patch size is used in all hole-filling processes. Experimental results demonstrate the effectiveness of the proposed method, which has better performance compared with previous methods in subjective and objective evaluation.

View synthesis is an effective method to generate the contents of multiple views based on a limited number of reference views, which can be used in 2D to 3D conversion, free viewpoint video and multiview video rendering. Depth-image-based rendering (DIBR) is a practical technique to generate virtual view by using a 2D reference view and its depth image. However, a critical problem in DIBR process is that disocclusions might be produced in the synthesized image because the background occluded by the foreground objects in the reference view may be exposed in the virtual view. In this paper, a local foreground removal method is proposed for disocclusion filling. Morphology-based depth image preprocessing is performed before DIBR, aiming to correct the depth value of the ghosts and remove ghost artifacts. In the synthesized virtual image, pixels on the disocclusion edge are identified and classified. Then they are positioned in the reference image by inverse 3D warping. Local foreground regions that occlude the corresponding background are removed from both the reference image and its depth image based on the disocclusion edge pixels. Removed region is filled with surrounding background contents, and depth information is used in this process to prevent foreground penetration. The predicted background contents are warped to the disocclusion region, thereby achieving the hole filling. Experimental results show that the proposed method performs better than the other methods in disocclusion filling, and improves the subjective and objective quality of the synthesized view. In the evaluation results of PSNR, SSIM, FSIMc and VSI, our method improves by 0.32-2.43dB, 0.0036-0.0155, 0.0041-0.0198 and 0.0012-0.0057 respectively compared with competitive methods.