Interactive, multi-user experiences are meant to define the present and future of Virtual Reality (VR). Such immersive experiences will typically consist of remote collaborations where content is streamed and/or synchronized over a network connection. Thus, real-time collaboration will be key. In this light, the responsiveness of the system and the network will define the overall experience. As such, understanding the effect of network distortions, especially related to time delay, on end-user’s perception (in terms of Quality-of-Experience (QoE)), performance, and collaboration becomes crucial. The existing literature, however, has mostly focused on network requirements from a system point-of-view, where the key performance parameters are only provided in the form of Quality-of-Service (QoS) parameters (such as end-to-end latency). However, the translation of these network impairments to the end-user experience is often omitted. The purpose of this paper is to fill the gap by providing a thorough investigation of the impact of latency on the perception of users while performing collaborative tasks in multi-user VR. To this end, an experimental framework was designed, developed, and tested. It is based on a multi-device synchronizing architecture, enabling two simultaneous users to work together in a gamified virtual environment. The developed test environment also allows for the identification of the most prominent network requirements and objective analysis for each traffic link. To experimentally investigate the impact of latency on user perception, a user study was conducted. Participants were paired and asked to perform the collaborative task under different latency-prone scenarios. The results show that users are able to easily distinguish between distorted and non-distorted network configurations. However, making a distinction between different manifestations of latency is much less straightforward. Moreover, factors such as the user’s role in the experience and the required task, and the level of interactivity and movement have an important influence on the subjective level of perception, the strength of the user’s preferences, and the occurrence of cybersickness. In contrast, no significant differences in objective metrics, such as system performance and user completion time were observed. These results can support the creation of collective QoE metrics that model the group as a whole rather than each individual separately. As such, this work provides an important step to dynamically counteract any drops in group dynamics and performance by means of smart interventions in the transmission system and/or virtual environment.

Point cloud streaming has recently attracted research attention as it has the potential to provide six degrees of freedom movement, which is essential for truly immersive media. The transmission of point clouds requires high-bandwidth connections, and adaptive streaming is a promising solution to cope with fluctuating bandwidth conditions. Thus, understanding the impact of different factors in adaptive streaming on the Quality of Experience (QoE) becomes fundamental. Point clouds have been evaluated in Virtual Reality (VR), where viewers are completely immersed in a virtual environment. Augmented Reality (AR) is a novel technology and has recently become popular, yet quality evaluations of point clouds in AR environments are still limited to static images. In this paper, we perform a subjective study of four impact factors on the QoE of point cloud video sequences in AR conditions, including encoding parameters (quantization parameters, QPs), quality switches, viewing distance, and content characteristics. The experimental results show that these factors significantly impact the QoE. The QoE decreases if the sequence is encoded at high QPs and/or switches to lower quality and/or is viewed at a shorter distance, and vice versa. Additionally, the results indicate that the end user is not able to distinguish the quality differences between two quality levels at a specific (high) viewing distance. An intermediate-quality point cloud encoded at geometry QP (G-QP) 24 and texture QP (T-QP) 32 and viewed at 2.5m can have a QoE (i.e., score 6.5 out of 10) comparable to a high-quality point cloud encoded at 16 and 22 for G-QP and T-QP, respectively, and viewed at a distance of 5 m. Regarding content characteristics, objects with lower contrast can yield better quality scores. Participants’ responses reveal that the visual quality of point clouds has not yet reached an immersion level as desired. The average QoE of the highest visual quality is less than 8 out of 10. There is also a good correlation between objective metrics (e.g., color Peak Signal-to-Noise Ratio (PSNR) and geometry PSNR) and the QoE score. Especially the Pearson correlation coefficients of color PSNR is 0.84. Finally, we found that machine learning models are able to accurately predict the QoE of point clouds in AR environments. The subjective test results and questionnaire responses are available on Github: https://github.com/minhkstn/QoE-and-Immersion-of-Dynamic-Point-Cloud.

One of the major obstacles to the widespread adoption of Virtual Reality (VR) is Cybersickness. It is a sense of physical discomfort akin to motion sickness experienced by the users either during or subsequent to VR utilization. Typically, it is detected through explicit methods such as self-reported questionnaires, which are not ideal for online and implicit monitoring of users’ well-being. This study tackles the challenge of implicitly detecting and measuring Cybersickness in VR through physiological signals. Therefore, we propose a multimodal approach that integrates physiological signals with self-reported measures. We utilize a mixed-methods design combining quantitative and qualitative analyses, using a roller coaster simulation as the experimental paradigm. The research analyzes physiological data collected from a group of 22 participants exposed to a simulated VR rollercoaster experience, through statistical analysis and machine learning algorithms. The physiological markers studied include Electroencephalograms (EEG), Electrodermal Activity, Blood Volume Pulse, and skin Temperature signals. Additionally, the study elucidates the complex interplay among physiological markers using Explainable AI (XAI). We found out significant correlations between high Simulator Sickness Questionnaire scores and physiological measures such as brain rhythms and EEG indices related to engagement, visual fatigue, and drowsiness, as well as heart rate variability. Moreover, the proposed machine learning model achieves an accuracy of 86.66 % in detecting elevated Cybersickness symptoms, which is higher than the accuracy achieved by existing techniques. Further exploration using the XAI technique confirms the elevated levels of drowsiness and reduced engagement in participants experiencing elevated Cybersickness. By providing a comprehensive, multimodal approach for quantitative assessment, this study fills a gap in the existing literature and paves the path for the development of adaptive systems that modulate their behavior based on physiological data.

Virtual Reality (VR) is gaining attention in various domains such as entertainment, industry, mental healthcare and VR training. Al- though most of these use-cases are still limited to single-user tasks, a lot of applications are heavily depending on multi-user collaboration. Existing multi-user VR systems are most often created in a classic server-client architecture, however, which induces unpredictable network behaviour which can affect the end-user's Quality-of-Experience (QoE) and performance. In addition, the interaction methods in these systems are often constrained to either traditional VR controllers or very use-case specific interaction methods, such that general purpose haptic gloves form a somewhat under-explored part of literature. Therefore, we (i) present a networked, distributed multi-user VR system with synchronization of environments over a low-bandwidth networked connection. In addition, we (ii) enhance the experience by adding haptic gloves to the system, which we compare to the traditional VR controllers in a subjective experiment. As a proof-of-concept, a use case is implemented in which two users have to prepare and bake a virtual pizza. The results show that high framerates (> 90 Frames Per Second (FPS)) can be obtained while keeping network throughput to a minimum ( < 1 Mbps). The accompanying user study shows that haptic gloves are preferred when immersiveness is the main emphasis of the virtual environment, while controllers are more suited when performance is in the center of attention. In objective terms, the applicability of haptic feedback is highly dependent on the task at hand.

The future of human interaction is virtual. Thus it will require effective collaboration on tasks among users in remote settings. eXtended Reality (XR) is playing a leading role in this transition, offering a realm where virtual collaboration becomes not just possible but essential in situations where physical presence is limited by risk, cost, or complexity. However, while networks are continuously evolving, they can still introduce unexpected impairments that potentially degrade the user perception, i.e., the Quality-of-Experience (QoE), of such Collaborative Virtual Reality (CVR) scenarios. In response to this challenge, this paper presents a demonstrator designed to explicitly showcase the effects of network conditions on CVR. Our platform, centered around a pizza-making game, allows for exploration of the real-time impact of different network parameters, such as packet delay, loss, and throttling on the user engagement and perception in CVR. The framework employs a combination of subjective, objective, and physiological assessments, including the capture of heart rate and skin conductivity, to gain comprehensive insights into user experiences. Our platform not only allows users to directly experience the impact of network impairments on CVR interactions but also provides initial evidence of how such distortions affect both subjective perceptions and objective performance metrics.