Virtual reality (VR) applications are typically developed within their own immersive digital worlds; therefore, virtual spaces are usually treated as discrete from the physical space where augmented and mixed reality users exist, which makes it difficult to combine these heterogeneous realities into an integrated extended reality (XR) environment. Along these lines, we propose a method that enables a user to geometrically register the virtual space within a VR application to a real space using a commodity camera in a workspace as an anchor point. We first investigate the mathematical aspect of the computational model for connecting the virtual space to the physical world. Then, we present a computational procedure that implements our proposed method with numerical accuracy and stability. As an application, we demonstrate that users of two VR systems from different vendors may collaborate within a shared real workspace while interacting with each other physically. The presented method provides a key mechanism for enabling XR users to leverage these immersive technologies by effectively using different realities within an integrated environment.
We propose Neural Crossbreed, a feed-forward neural network that can learn a semantic change of input images in a latent space to create the morphing effect. Because the network learns a semantic change, a sequence of meaningful intermediate images can be generated without requiring the user to specify explicit correspondences. In addition, the semantic change learning makes it possible to perform the morphing between the images that contain objects with significantly different poses or camera views. Furthermore, just as in conventional morphing techniques, our morphing network can handle shape and appearance transitions separately by disentangling the content and the style transfer for rich usability. We prepare a training dataset for morphing using a pre-trained BigGAN, which generates an intermediate image by interpolating two latent vectors at an intended morphing value. This is the first attempt to address image morphing using a pre-trained generative model in order to learn semantic transformation. The experiments show that Neural Crossbreed produces high quality morphed images, overcoming various limitations associated with conventional approaches. In addition, Neural Crossbreed can be further extended for diverse applications such as multi-image morphing, appearance transfer, and video frame interpolation.
A triangular mesh obtained by scanning 3D models typically contains holes. We present an effective technique for filling a hole in a triangular mesh in geometric modeling. Simple triangulation of a hole is refined and remeshed iteratively to generate an initial patch. The generated patch is then enhanced to become a target patch by minimizing the variation of principal curvatures. In discrete approximation, this produces a third-order Laplacian system of sparse symmetric positive definite matrix, and the symmetry can efficiently be used to find the robust solutions to the given Laplacian system. Laplacian smoothing of the target patch is defined as a source patch. The shape difference between two corresponding vertices of the source and the target patches is measured in terms of Euclidean distance and curvature variation. On the basis of the shape difference and a user-specified control parameter, different blending weights are determined for each vertex, and the final patch is generated by blending two patches. We demonstrate the effectiveness of our technique by discussing several examples. The experimental results show that our technique can effectively restore salient geometric features of the original shape.
Cluster computing has attracted much attention as an effective way of solving large-scale problems. However, only a few attempts have been made to explore mobile computing clusters that can be easily built using commodity smartphones and tablets. To investigate the possibility of mobile cluster-based rendering of large datasets, we developed a mobile GPU ray tracer that renders nontrivial 3D scenes with many millions of triangles at an interactive frame rate on a small-scale mobile cluster. To cope with the limited processing power and memory space, we first present an effective 3D scene representation scheme suitable for mobile GPU rendering. Then, to avoid performance impairment caused by the high latency and low bandwidth of mobile networks, we propose using a static load balancing strategy, which we found to be more appropriate for the vulnerable mobile clustering environment than a dynamic strategy. Our mobile distributed rendering system achieved a few frames per second when ray tracing 1024 × 1024 images, using only 16 low-end smartphones, for large 3D scenes, some with more than 10 million triangles. Through a conceptual demonstration, we also show that the presented rendering scheme can be effectively explored for augmenting real scene images, captured or perceived by augmented and mixed reality devices, with high quality ray-traced images.