Rehabilitation for stroke afflicted patients, through exercises tailored for individual needs, aims at relearning basic motor skills, especially in the extremities. Rehabilitation through Augmented Reality (AR) based games engage and motivate patients to perform exercises which, otherwise, maybe boring and monotonic. Also, mirror therapy allows users to observe one's own movements in the game providing them with good visual feedback. This paper presents an augmented reality based system for rehabilitation by playing four interactive, cognitive and fun Exergames (exercise and gaming). The system uses low-cost RGB-D cameras such as Microsoft Kinect V2 to capture and generate 3D model of the person by extracting him/her from the entire captured data and immersing it in different interactive virtual environments. Animation based limb movement enhancement along with cognitive aspects incorporated in the game can help in positive reinforcement, progressive challenges and motion improvement. Recording module of the toolkit allows future reference and facilitates feedback from the physician. 10 able-bodied users, 2 psychological experts and 2 Physical Medicine and Rehabilitation physicians evaluated the user experience and usability aspects of the exergames. Results obtained shows the games to be fun and realistic, and at the same time engaging and motivating for performing exercises.

Training through Multi-modal Collaborative Virtual Laboratory (MMCVL) aims at developing or improving skills of students without the need of them being physically present. MMCVL addresses the problems faced by the current education system such as lack of resources and safe use of expensive laboratory equipment. It also provides a means for easy collaboration between students and teachers from different universities and schools. Multimedia data in terms of 3D mesh, skeleton, audio and virtual object physics interaction are incorporated in MMCVL to provide a 3D Tele-Immersion environment. The system uses low-cost RGB-D cameras such as Microsoft Kinect V2 to capture and generate 3D model of the person by extracting him/her from the entire captured data and immersing it in different interactive virtual environments. Different lab environments can be virtually created to provide for and replace the real laboratories. Object interactions allow for users to interact with the virtual objects present in the scene similar to real lab setup. Recording module of the toolkit allows future reference for training and testing of the student. The system also facilitates virtual feedback by giving the correct measurements, video demonstrations as well as audio communication with the collaborating teacher. Two different Chemistry experiments were performed to evaluate the user experience and usability aspects of MMCVL. Results obtained from the user study analysis prove the system to be fun and realistic, and at the same time engaging and motivating for learning laboratory experiments.

To describe a novel 3-dimensional (3D) exergames system and the results of a clinical feasibility study of stroke survivors needing in-home rehabilitation.The customisable Personalized In-home eXEr...

Users' Quality of Experience (QoE) in Interactive 3D Tele-Immersion (i3DTI) systems is influenced by several factors such as the quality of the "live" 3D avatars of the users, network latency, rendering methodology (head mounted display or regular TV type of display), etc. Hence, it becomes important to answer the question: "Is Visual Quality (VQ) the only factor to be considered or do better immersion and faster interactions matter, in having good QoE?" To answer this question, in this paper, a highly optimized state-of-the-art i3DTI framework implementation is introduced along with a soccer-inspired penalty shootout game. This game allows users to experience various situations, view different angles, perceive delays, track virtual ball motion, and play naturally using their entire bodies. A head mounted display device - Oculus Rift allows the users to get completely immersed and perform better in the penalty shootout game compared to watching themselves play on a 3D TV. This scenario is obtained in a controlled lab setting with ultra-high-speed network that has ultra-low latency, high VQ, fast and realistic interactions. Such ideal conditions are not typically available in a wide area Internet. Hence, for faster interaction scenario, we used lesser RGB-D cameras for 3D reconstruction, thereby reducing the model quality significantly. The high responsiveness of the game masked the user's perception of quality, and resulted in them not noticing the lower VQ of the reconstructed 3D models. Based on the results from the user study that focused on immersion, interaction and VQ aspects; users felt the game to be visually appealing, intuitive, engaging, and highly entertaining in all of the scenarios.

RGB-D cameras, such as the Microsoft Kinect, provide us with the 3D information, color and depth, associated with the scene. Interactive 3D Tele-Immersion (i3DTI) systems use such RGB-D cameras to capture the person present in the scene in order to collaborate with other remote users and interact with the virtual objects present in the environment. Using a single camera, it becomes difficult to estimate an accurate skeletal pose and complete 3D model of the person, especially when the person is not in the complete view of the camera. With multiple cameras, even with partial views, it is possible to get a more accurate estimate of the skeleton of the person leading to a better and complete 3D model. In this paper, we present a real-time skeletal pose identification approach that leverages on the inaccurate skeletons of the individual Kinects, and provides a combined optimized skeleton. We estimate the Probability of an Accurate Joint (PAJ) for each joint from all of the Kinect skeletons. We determine the correct direction of the person and assign the correct joint sides for each skeleton. We then use a greedy consensus approach to combine the highly probable and accurate joints to estimate the combined skeleton. Using the individual skeletons, we segment the point clouds from all the cameras. We use the already computed PAJ values to obtain the Probability of an Accurate Bone (PAB). The individual point clouds are then combined one segment after another using the calculated PAB values. The generated combined point cloud is a complete and accurate 3D representation of the person present in the scene. We validate our estimated skeleton against two well-known methods by computing the error distance between the best view Kinect skeleton and the estimated skeleton. An exhaustive analysis is performed by using around 500000 skeletal frames in total, captured using 7 users and 7 cameras. Visual analysis is performed by checking whether the estimated skeleton is completely present within the human model. We also develop a 3D Holo-Bubble game to showcase the real-time performance of the combined skeleton and point cloud. Our results show that our method performs better than the state-of-the-art approaches that use multiple Kinects, in terms of objective error, visual quality and real-time user performance.