Virtual training systems are in an increasing demand because of real-world training, which requires a high cost or accompanying risk, and can be conducted safely through virtual environments. For virtual training to be effective for users, it is important to provide realistic training situations; however, virtual reality (VR) content using VR controllers for experiential learning differ significantly from real content in terms of tangible interactions. In this paper, we propose a method for enhancing the presence and immersion during virtual training by applying various sensors to tangible virtual training as a way to track the movement of real tools used during training and virtualizing the entire body of the actual user for transfer to a virtual environment. The proposed training system connects virtual and real-world spaces through an actual object (e.g., an automobile) to provide the feeling of actual touch during virtual training. Furthermore, the system measures the posture of the tools (steam gun and mop) and the degree of touch and applies them during training (e.g., a steam car wash.) User-testing is conducted to validate the increase in the effectiveness of virtual job training.

In this paper, we present a novel approach for 3D human pose estimation using depth data from egocentric viewpoints. Depth data has the advantage that it is less sensitive to color and lighting changes. We acquired depth data streamed from multiple depth cameras attached to a user’s head and calibrated them into a depth map. For joint detection, a ResNet-based network was optimized with the skeletal joints of a Kinect camera. Unlike previous approaches, the proposed approach can track 3D human poses in an egocentric setup with a small dataset.