This paper presents a mixed reality (MR) system that results from the integration of a telepresence system and an application to improve collaborative space exploration. The system combines free viewpoint video with immersive projection technology to support nonverbal communication (NVC), including eye gaze, interpersonal distance, and facial expression. Importantly, these features can be interpreted together as people move around the simulation, maintaining a natural social distance. The application is a simulation of Mars, within which the collaborators must come to agreement over; for example, where the Rover should land and go. The first contribution is the creation of an MR system supporting contextualization of NVC. Two technological contributions are prototyping a technique to subtract a person from a background that may contain physical objects and/or moving images and a lightweight texturing method for multiview rendering, which provides balance in terms of visual and temporal quality. A practical contribution is the demonstration of pragmatic approaches to sharing space between display systems of distinct levels of immersion. A research tool contribution is a system that allows comparison of conventional authored and video-based reconstructed avatars, within an environment that encourages exploration and social interaction. Aspects of system quality, including the communication of facial expression and end-to-end latency are reported.

The emergence of off-screen interaction devices is bringing the field of virtual reality to a broad range of applications where virtual objects can be manipulated without the use of traditional peripherals. However, to facilitate object interaction, other stimuli such as haptic feedback are necessary to improve the user experience. To enable the identification of virtual 3D objects without visual feedback, a haptic display based on a vibrotactile glove and multiple points of contact gives users an enhanced sensation of touching a virtual object with their hands. Experimental results demonstrate the capacity of this technology in practical applications.

Estimation of human emotions plays an important role in the development of modern brain-computer interface devices like the Emotiv EPOC+ headset. In this paper, we present an experiment to assess the classification accuracy of the emotional states provided by the headset’s application programming interface (API). In this experiment, several sets of images selected from the International Affective Picture System (IAPS) dataset are shown to sixteen participants wearing the headset. Firstly, the participants’ responses in form of a self-assessment manikin questionnaire to the emotions elicited are compared with the validated IAPS predefined valence, arousal and dominance values. After statistically demonstrating that the responses are highly correlated with the IAPS values, several artificial neural networks (ANNs) based on the multilayer perceptron architecture are tested to calculate the classification accuracy of the Emotiv EPOC+ API emotional outcomes. The best result is obtained for an ANN configuration with three hidden layers, and 30, 8 and 3 neurons for layers 1, 2 and 3, respectively. This configuration offers 85% classification accuracy, which means that the emotional estimation provided by the headset can be used with high confidence in real-time applications that are based on users’ emotional states. Thus the emotional states given by the headset’s API may be used with no further processing of the electroencephalogram signals acquired from the scalp, which would add a level of difficulty.

The use of haptics in virtual reality and video games has received growing attention as a means of enhancing the sensation of immersion in these environments. The sensation of touching virtual objects not only augments the impression of reality but also can improve the performance. However, the design of haptic interactions is not an easy task, and it usually needs a great effort due to the absence of powerful prototyping toolkits. Thus, this article proposes a vibrotactile prototyping toolkit for Eccentric Rotating Mass actuators named VITAKI. The main objective of this platform is to facilitate the prototyping and testing procedures of new vibrotactile interaction techniques for Virtual Reality and video games. A detailed description of the design of the system is provided, presenting the hardware and software elements that make up the VITAKI toolkit. In addition, its application to two different examples to illustrate its use is provided. Finally, a preliminary evaluation of this toolkit is presented. This evaluation is divided into two main stages. On one hand, a study of Olsen’s criteria is performed to analyze its general capabilities. On the other hand, a comparison with previously presented proposals is included too. These two analyses, together with other experiments where the devices created with the toolkit were tested by end users, highlight its main features and its advantages over other proposals.

With product life-cycles getting shorter and limited availability of natural resources, the paradigm shift towards the circular economy is being impulsed. In this domain, the successful adoption of remanufacturing is key. However, its associated process efficiency is to date limited given high flexibility requirements for product disassembly. With the emergence of Industry 4.0, natural human-robot interaction is expected to provide numerous benefits in terms of (re)manufacturing efficiency and cost. In this regard, vision-based and wearable-based approaches are the most extended when it comes to establishing a gesture-based interaction interface. In this work, an experimental comparison of two different movement-estimation systems—(i) position data collected from Microsoft Kinect RGB-D cameras and (ii) acceleration data collected from inertial measurement units (IMUs)—is addressed. The results point to our IMU-based proposal, OperaBLE, having recognition accuracy rates up to 8.5 times higher than these of Microsoft Kinect, which proved to be dependent on the movement’s execution plane, subject’s posture, and focal distance.

People with schizophrenia have difficulty recognizing the emotions in the facial expressions of others, which affects their social interaction and functioning in the community. Static stimuli such as photographs have been used traditionally to examine deficiencies in the recognition of emotions in patients with schizophrenia, which has been criticized by some authors for lacking the dynamism that real facial stimuli have. With the aim of overcoming these drawbacks, in recent years, the creation and validation of virtual humans has been developed. This work presents the results of a study that evaluated facial recognition of emotions through a new set of dynamic virtual humans previously designed by the research team, in patients diagnosed of schizophrenia. The study included 56 stable patients, compared with 56 healthy controls. Our results showed that patients with schizophrenia present a deficit in facial affect recognition, compared to healthy controls (average hit rate 71.6% for patients vs 90.0% for controls). Facial expressions with greater dynamism (compared to less dynamic ones), as well as those presented from frontal view (compared to profile view) were better recognized in both groups. Regarding clinical and sociodemographic variables, the number of hospitalizations throughout life did not correlate with recognition rates. There was also no correlation between functioning or quality of life and recognition. A trend showed a reduction in the emotional recognition rate as a result of increases in Positive and Negative Syndrome Scale (PANSS), being statistically significant for negative PANSS. Patients presented a learning effect during the progression of the task, slightly greater in comparison to the control group. This finding is relevant when designing training interventions for people with schizophrenia. Maintaining the attention of patients and getting them to improve in the proposed tasks is a challenge for today’s psychiatry.