Effective development of virtual reality (VR) applications is heavily reliant on the evaluation of user experience (UX). However, traditional methods such as questionnaires have inherent limitations which hinder their ability to capture nuanced behavioral responses and impede the agility of VR content creation: They are time-consuming, burdensome for users, and require significant human effort for interpretation. This study introduces an automated framework aimed at addressing the limitations of questionnaire-based evaluations to assess UX in VR. Our primary focus is to validate the concept of this framework through assessment of the sense of presence (SOP), a crucial psychological perception with significant impact on VR UX. Our proposed framework utilizes a deep neural network (DNN) to analyze patterns in multimodal behavioral cues, including facial expressions, head movements, and hand movements, to predict scores from the Igroup Presence Questionnaire (IPQ). Additionally, we introduce two statistical profiles: the Visual Entropy Profile (VEP), which offers insights into visual complexity by depicting scene entropy, and the Experiential Presence Profile (EPP), which is designed to capture users’ historical SOP levels to enable personalized baseline and sensitivity estimation. The proposed framework achieves a significant correlation between actual and predicted IPQ scores, with a Spearman’s rank correlation coefficient of 0.7303, showcasing the potential of DNNs in analyzing complex behavioral signals and automating SOP assessment. This study represents a pioneering effort in leveraging DNNs for the automatic assessment of SOP and paves the way for future advances in automatically assessing VR UX and unlocking new opportunities in the field.

We present ElectroAR, a visual and tactile sharing system for hand skills training. This system comprises a head-mounted display (HMD), two cameras, a tactile sensing glove, and an electro-tactile stimulation glove. The trainee wears the tactile sensing glove that gets pressure data from touching different objects. His/her movements are recorded by two cameras, which are located in front and top side of the workspace. In the remote site, the trainer wears the electro-tactile stimulation glove. This glove transforms the remotely collected pressure data to electro-tactile stimuli. Additionally, the trainer wears an HMD to see and guide the movements of the trainee. The key part of this project is to combine distributed tactile sensor and electro-tactile display to let the trainer understand what the trainee is doing. Results show our system supports a higher user’s recognition performance.

In user experience evaluation (UXE), it is generally accepted that the order in which users perform tasks when using a product is often random rather than fixed. UXE based on these so-called randomly ordered tasks is challenging. Although several articles have been published on UXE, none have proposed a technique to evaluate the significance of randomly ordered tasks. In this study, we propose a new approach to predict final user satisfaction based on UX related to randomly ordered tasks. We aimed to study the importance of task order in the UX. In the main experiment, 60 participants completed questionnaires about satisfaction while performing a series of tasks on a travel agency website. Among the machine learning models tested, we found that accounting for the order or sequence of actions actually performed by users in a support vector machine (SVM) algorithm with a polynomial kernel produced the most accurate predictions of final user satisfaction (97%). These findings indicate that some machine learning techniques can comprehend participants’ randomly ordered UX data. Moreover, using random ordering, which accounts for the actual order of actions performed by users, can significantly impact the prediction of final user satisfaction.