This manuscript identifies and documents unsolved problems and research challenges in the extended reality (XR) domain (i.e., virtual (VR), augmented (AR), and mixed reality (MR)). The manuscript is structured to include technology, design, and human factor perspectives. The text is visualization/display-focused, that is, other modalities such as audio, haptic, smell, and touch, while important for XR, are beyond the scope of this paper. We further narrow our focus to mainly geospatial research, with necessary deviations to other domains where these technologies are widely researched. The main objective of the study is to provide an overview of broader research challenges and directions in XR, especially in spatial sciences. Aside from the research challenges identified based on a comprehensive literature review, we provide case studies with original results from our own studies in each section as examples to demonstrate the relevance of the challenges in the current research. We believe that this paper will be of relevance to anyone who has scientific interest in extended reality, and/or uses these systems in their research.

During evacuation, individual navigation behavior is often dictated by the legibility of evacuation signs and the environmental setting itself. People tend to follow previously-used and known routes (to retrace) rather than follow evacuation signage. This has proven undesirable, even fatal, in emergencies and such behavior calls for a better understanding of the influencing factors. This contribution consists of a virtual reality experiment in which 72 participants evacuated from a hotel building in which the spatial configurations were altered. The tendency to retrace diminished when the evacuation route led through a wider, straight corridor, suggesting that the building’s spatial configuration has an important influence on retracing and should be more carefully considered in the design of buildings in and agent-based simulations.

The building design is a crucial factor that can be actively adjusted and optimized to prevent human and property threats in emergency scenarios. Previous research suggests that specific building layouts may significantly influence human behaviour during evacuation. However, detailed empirical data about human behaviour in various types of buildings with different layouts are still missing and only marginal recommendations from this field are reflected in actual construction practice. In this study, desktop VR technologies were employed to study human decision-making in problematic T-intersections in the context of an emergency evacuation. More specifically, we studied fundamental attributes of buildings such as the width and length of the corridors and the presence of stairs to explore how they influence the choice of the evacuation route. The space-syntax isovist method was used to describe spatial parameters of corridors, which makes the results applicable to all buildings. Behavioural data from 208 respondents were analysed using multilevel regression models. Our results support previous claims concerning the importance of specific spatial layouts of evacuation corridors because respondents systematically chose wider and shorter corridors with visible staircases as the preferred evacuation route. The present findings further promote the ongoing discussion on the design of marked evacuation routes and building design that takes human factors into consideration.

Agent-based evacuation modeling represents an effective tool for making predictions about evacuation aspects of buildings such as evacuation times, congestions, and maximum safe building capacity. Collection of real behavioral data for calibrating agent-based evacuation models is time-consuming, costly, and completely impossible in the case of buildings in the design phase, where predictions about evacuation behavior are especially needed. In recent years evacuation experiments conducted in virtual reality (VR) have been frequently proposed in the literature as an effective tool for collecting data about human behavior. However, empirical studies which would assess validity of VR-based data for such purposes are still rare and considerably lacking in the agent-based evacuation modeling domain. This study explores opportunities that the VR behavioral data may bring for refining outputs of agent evacuation models. To this end, this study employed multiple input settings of agent-based evacuation models (ABEMs), including those based on the data gathered from the VR evacuation experiment that mapped out evacuation behaviors of individuals within the building. Calibration and evaluation of models was based on empirical data gathered from an original evacuation exercise conducted in a real building (N = 35) and its virtual twin (N = 38). This study found that the resulting predictions of single agent models using data collected in the VR environment after proposed corrections have the potential to better predict real-world evacuation behavior while offering desirable variance in the data outputs necessary for practical applications.

This work addresses the impact of a geovisualization’s level of realism on a user’s experience in indoor navigation. The key part of the work is a user study in which participants navigated along a designated evacuation route previously learnt in a virtual tour or traditional 2D floor plan. The efficiency and effectiveness of completing the task was measured by the number of incorrect turns during navigation and completion time. The complexity of mental spatial representations that participants developed before and after navigating the route was also evaluated. The data was obtained using several qualitative and quantitative research methods (mobile eye tracking, structured interviews, sketching of cognitive maps, creation of navigation instructions, and additional questions to evaluate spatial orientation abilities). A total of 36 subjects (17 in the “floor plan” group and 19 in the “virtual tour” group) participated in the study. The results showed that the participants from both groups were able to finish the designated navigation route, but more detailed mental spatial representations were developed by the “virtual tour” group than the “floor plan” group. The participants in the virtual tour group created richer navigation instructions both before and after evacuation, mentioned more landmarks and could recall their characteristics. Visual landmark characteristics available in the virtual tour also seemed to support the correct decision-making.

From the user perspective, 3D geospatial data visualizations are one of the bridges between the physical and the digital world. As such, the potential of 3D geospatial data visualizations is frequently discussed within and beyond the digital twins. The effects on human cognitive processes in complex spatial tasks is rather poorly known. No uniform standards exist for the 3D technologies used in these tasks. Although stereoscopic geovisualizations presented using 3D technologies enhance depth perception, it has been suggested that the visual discomfort experienced when using 3D technology outweighs its benefits and results in lower efficiency and errors. In the present study, sixty participants using 3D technologies were tested in terms of their ability to make informed decisions in selecting the correct position of a virtual transmitter in a digital twin and a digital terrain model, respectively. Participants (n = 60) were randomly assigned into two groups, one using 3D technology engaging stereoscopic shutter glasses and the second working with standard computer screen-based visualizations. The results indicated that the participants who used shutter glasses performed significantly worse in terms of response time (W = 175.0; p < 0.001, r = −0.524). This finding verifies previous conclusions concerning the unsuitability of stereoscopic visualization technology for complex decision-making in geospatial tasks.