In this paper, we propose a multi-view stereo matching method, EnSoft3D (Enhanced Soft 3D Reconstruction) to obtain dense and high-quality depth images. Multi-view stereo is one of the high-interest research areas and has wide applications. Motivated by the Soft3D reconstruction method, we introduce a new multi-view stereo matching scheme. The original Soft3D method is introduced for novel view synthesis, while occlusion-aware depth is also reconstructed by integrating the matching costs of the Plane Sweep Stereo (PSS) and soft visibility volumes. However, the Soft3D method has an inherent limitation because the erroneous PSS matching costs are not updated. To overcome this limitation, the proposed scheme introduces an update process of the PSS matching costs. From the object surface consensus volume, an inverse consensus kernel is derived, and the PSS matching costs are iteratively updated using the kernel. The proposed EnSoft3D method reconstructs a highly accurate 3D depth image because both the multi-view matching cost and soft visibility are updated simultaneously. The performance of the proposed method is evaluated by using structured and unstructured benchmark datasets. Disparity error is measured to verify 3D reconstruction accuracy, and both PSNR and SSIM are measured to verify the simultaneous enhancement of view synthesis.

In the multiview stereo (MVS) vision, it is difficult to estimate accurate depth in the textureless and occluded regions. To solve this problem, several MVS investigations employ the matching cost volume (MCV) approach to refine the cost in the textureless and occluded regions. Usually, the matching costs in the large textureless image regions are not reliable. In addition, if an occluded region is also textureless, the matching cost contains a significant error. The goal of the proposed MVS method is to reconstruct accurate depth maps in both large-textureless and occluded-textureless regions by iteratively updating the erroneous disparity. The erroneous disparity in the textureless region is updated by the 3D disparity plane of the region in the inverse-depth space. Then, the surface consensus is computed and used to run the two processes, the surface consensus refinement and the matching cost update. By the iterative update of the 3D inverse depth plane, surface consensus, and matching cost, the performance of the depth reconstruction in the large-textureless and occluded-textureless regions is greatly improved. The performance of the proposed method is analyzed using the Middlebury multiview stereo dataset. The depth reconstruction performance is also compared with several stereo vision methods.

We propose a multiview stereo (MVS) method that is based on the forward and backward propagation of matching costs for the incremental refinement of multiview disparity maps. The volume-based MVS method requires numerous three-dimensional (3D) memory volumes to compute, store, and process the stereo matching costs. However, owing to memory limitations, conventional MVS methods allocate the memory of the 3D cost volumes only to the reference and its neighbor views. Thus, each reference view can only use the matching costs from a limited number of neighbor views. This study addresses this inherent MVS problem by employing a forward and backward cost propagation (FBCP) scheme. First, a subpart of the input views is used to obtain disparity maps with a dense MVS method. Once all matching costs of the subpart views are sufficiently refined, the FBCP is performed for a new neighbor view. Immediately after the cost volume of the new view is computed, all matching costs of the subpart are forward propagated and fused with the initial cost of the new view. Furthermore, the new fused cost is backward propagated into the subpart to refine the previous costs again using the new fused cost. All cost volumes can be incrementally computed and refined without any limitation on the number of views using the proposed FBCP scheme. In the final step, all disparity maps are obtained from the refined cost volumes and fused into single point clouds. Moreover, we propose the use of surface consensus to obtain accurate fused point clouds for the fusion of the disparity maps. The performance of the proposed method is evaluated using the fused point clouds. The mean error distance and percentage within the threshold are compared to the ground truth.