A virtual reality (VR) environment is different from the generally produced video-based experience, as it creates an interactive user experience (UX) by allowing the users to respond to operations or commands using various input devices. In particular, as existing analog-type fashion shows that have limitations in space and time have evolved into digital fashion shows that include interaction, the importance of the naturalness of interaction and visual expression is being emphasized in UX research. However, study cases that maximize user immersion through the interaction of visual changes in the stage and clothes of a digital fashion show are scarce. As such, this study introduces an interactive VR fashion show to analyze the impact of interaction on the UX in a VR environment. In the design of the interactive VR fashion show, various interaction elements that affect the UX are selected and the scope of UX application is suggested. In addition, by utilizing a commercial game engine, the production process necessary to simulate an existing fashion show is shown step-by-step. The user test in this study is examined in-depth by dividing the evaluation of the user into a two-step survey, and the results are objectively verified through a statistical analysis.

Dynamic human movements such as dance are difficult to capture without using external markers due to the high complexity of a dancer’s body. This article introduces a marker-free motion capture and composition system for dance motion that uses multiple RGB and depth sensors. Our motion capture system utilizes a set of high-speed RGB and depth sensors to generate skeletal motion data from an expert dancer. During the motion acquisition process, a skeleton tracking method based on a particle filter is provided to estimate the motion parameters for each frame from a sequence of color images and depth features retrieved from the sensors. The expert motion data become archived in a database. The authoring methods in our composition system automate most of the motion editing processes for general users by providing an online motion search with an input posture and then performing motion synthesis on an arbitrary motion path. Using the proposed system, we demonstrate that various dance performances can be composed in an intuitive and efficient way on client devices such as tablets and kiosk PCs.

Human poses are difficult to estimate due to the complicated body structure and the self-occlusion problem. In this paper, we introduce a marker-less system for human pose estimation by detecting and tracking key body parts, namely the head, hands, and feet. Given color and depth images captured by multiple red, green, blue, and depth (RGB-D) cameras, our system constructs a graph model with segmented regions from each camera and detects the key body parts as a set of extreme points based on accumulative geodesic distances in the graph. During the search process, local detection using a supervised learning model is utilized to match local body features. A final set of extreme points is selected with a voting scheme and tracked with physical constraints from the unified data received from the multiple cameras. During the tracking process, a Kalman filter-based method is introduced to reduce positional noises and to recover from a failure of tracking extremes. Our system shows an average of 87% accuracy against the commercial system, which outperforms the previous multi-Kinects system, and can be applied to recognize a human action or to synthesize a motion sequence from a few key poses using a small set of extremes as input data.

In this paper, we introduce a novel acoustic source localization in a three-dimensional (3D) space, based on a direction estimation technique. Assuming an acoustic source at a distance from adjacent microphones, its waves spread in a planar form called a planar wavefront. In our system, the directions and steering angles between the acoustic source and the microphone array are estimated based on a planar wavefront model using a delay and sum beamforming (DSBF) system and an array of two-dimensional (2D) microelectromechanical system (MEMS) microphones. The proposed system is designed with parallel processing hardware for real-time performance and implemented using a cost-effective field programmable gate array (FPGA) and a micro control unit (MCU). As shown in the experimental results, the localization errors of the proposed system were less than 3 cm when an impulsive acoustic source was generated over 1 m away from the microphone array, which is comparable to a position-based system with reduced computational complexity. On the basis of the high accuracy and real-time performance of localizing an impulsive acoustic source, such as striking a ball, the proposed system can be applied to screen-based sports simulation.