In stereoscopic television, there is a trade-off between visual comfort and 3-dimensional (3D) impact with respect to the baseline-stretch of a 3DTV camera. It is necessary to adjust the baseline-stretch at an appropriate distance depending on the contents of a scene if we want to obtain a subjectively optimal quality of an image. However, it is very hard to obtain a small baseline-stretch using commercially available cameras of broadcasting quality where the sizes of the lens and CCD module are large. In order to overcome this limitation, we attempt to freely control the baseline-stretch of a stereoscopic camera by synthesizing the virtual views at the desired location of interval between two cameras. This proposed technique is based on the stereo matching and view synthesis techniques. We first obtain a dense disparity map using a hierarchical stereo matching with the edge-adaptive multiple shifted windows. Then, we synthesize the virtual views using the disparity map. Simulation results with various stereoscopic images demonstrate the effectiveness of the proposed technique.
In this paper, we propose a multi-view stereo matching method, EnSoft3D (Enhanced Soft 3D Reconstruction) to obtain dense and high-quality depth images. Multi-view stereo is one of the high-interest research areas and has wide applications. Motivated by the Soft3D reconstruction method, we introduce a new multi-view stereo matching scheme. The original Soft3D method is introduced for novel view synthesis, while occlusion-aware depth is also reconstructed by integrating the matching costs of the Plane Sweep Stereo (PSS) and soft visibility volumes. However, the Soft3D method has an inherent limitation because the erroneous PSS matching costs are not updated. To overcome this limitation, the proposed scheme introduces an update process of the PSS matching costs. From the object surface consensus volume, an inverse consensus kernel is derived, and the PSS matching costs are iteratively updated using the kernel. The proposed EnSoft3D method reconstructs a highly accurate 3D depth image because both the multi-view matching cost and soft visibility are updated simultaneously. The performance of the proposed method is evaluated by using structured and unstructured benchmark datasets. Disparity error is measured to verify 3D reconstruction accuracy, and both PSNR and SSIM are measured to verify the simultaneous enhancement of view synthesis.
16 April 2025 in IEEE Transactions on Image Processing
A new approach for occlusion-robust 3D human mesh reconstruction from a single image is introduced in this paper. Since occlusion has emerged as a major problem to be resolved in this field, there have been meaningful efforts to deal with various types of occlusions (e.g., person-to-person occlusion, person-to-object occlusion, self-occlusion, etc.). Although many recent studies have shown the remarkable progress, previous regression-based methods still have respective limitations to handle occlusion problems due to the lack of the appearance information. To address this problem, we propose a novel method for human mesh reconstruction based on the pose-relevant subspace analysis. Specifically, we first generate a set of eigenvectors, so-called eigenposes, by conducting the singular value decomposition (SVD) of the pose matrix, which contains diverse poses sampled from the training set. These eigenposes are then linearly combined to construct a target body pose according to fusing coefficients, which are learned through the proposed network. Such combination of principal body postures (i.e., eigenposes) in a global manner gives a great help to cope with partial ambiguities by occlusions. Furthermore, we also propose to exploit a joint injection module that efficiently incorporates the spatial information of visible joints into the encoded feature during the estimation process of fusing coefficients. Experimental results on benchmark datasets demonstrate the ability of the proposed method to robustly reconstruct the human mesh under various occlusions occurring in real-world scenarios. The code and model are publicly available at: https://github.com/DCVL-3D/Eigenpose_release.
17 October 2022 in IEEE Transactions on Multimedia
Various studies have been conducted on instance segmentation and made great strides over the past few years. Most recently, instance-specific mask generation via dynamic kernel predictions has shown the significant performance improvement even without bounding boxes as well as anchors. However, this scheme still does not fully consider dynamic properties since the size of the receptive field is not enough to cover the spatially-meaningful range due to memory limitations. Furthermore, the single-fused feature often fails to grasp complicated boundaries for objects of different sizes. In this paper, we propose the dynamic residual filtering method with the Laplacian pyramid, which separately restores the global layout and local boundaries of instance masks. Specifically, we firstly apply the Laplacian pyramid-based decomposition scheme to features encoded from the backbone and subsequently restore sub-band mask residuals from coarse to fine pyramid levels. To do this, we design spatially-aware convolution filters to progressively capture the residual form of mask features at each level of the Laplacian pyramid while holding deformable receptive fields with dynamic offset information. This is fairly desirable since global and local properties of mask features can be accurately restored with keeping the spatial flexibility through the invertible process of the Laplacian reconstruction. Experimental results on the COCO dataset demonstrate that our proposed method achieves the state-of-the-art performance, i.e., 42.7% AP. The code and model are publicly available at: https://github.com/tjqansthd/LapMask .
We present a novel multi-view video replay service system using multi-PTZ cameras. Proposed system firstly captures multi-view videos for an object of interest with converged multi-cameras and then synthesizes intermediate views by grid mesh-based warping at a certain time instant. Next, captured multi-camera videos and synthesized intermediate view videos are combined to generate a multi-view replay video at a certain time instant. A generated multi-view replay video is transmitted to the web-based multi-view video player and it can be showed to the viewers selectively with multi-camera videos. Test experiments verified the successful operation of the proposed system.
About Me
218 Gajeong-ro, Yuseong-gu, Daejeon, Republic of Korea
218 Gajeong-ro, Yuseong-gu, Daejeon, Republic of Korea