This manuscript identifies and documents unsolved problems and research challenges in the extended reality (XR) domain (i.e., virtual (VR), augmented (AR), and mixed reality (MR)). The manuscript is structured to include technology, design, and human factor perspectives. The text is visualization/display-focused, that is, other modalities such as audio, haptic, smell, and touch, while important for XR, are beyond the scope of this paper. We further narrow our focus to mainly geospatial research, with necessary deviations to other domains where these technologies are widely researched. The main objective of the study is to provide an overview of broader research challenges and directions in XR, especially in spatial sciences. Aside from the research challenges identified based on a comprehensive literature review, we provide case studies with original results from our own studies in each section as examples to demonstrate the relevance of the challenges in the current research. We believe that this paper will be of relevance to anyone who has scientific interest in extended reality, and/or uses these systems in their research.

Computer-based evacuation simulations are important tools for emergency managers. These simulations vary in complexity and include 2D and 3D GIS-based network analyses, agent-based models, and sophisticated models built on documented human behaviour and particle dynamics. Despite the influential role of built environments in determining human movement, a disconnect often exists between the features of the real world and the way they are represented within these simulation environments. The proliferation of emergency management location-aware mobile devices, along with a recent infatuation for augmented reality (AR), has resulted in new wayfinding and hazard assessment tools that bridge this gap, allowing users to visualize geospatial information superimposed on the real world. In this paper, we report research and development that has produced AR geovisual analytical systems, enabling visual analysis of human dynamics in multi-level built environments with complex thoroughfare network infrastructure. We demonstrate prototypes that show how mixed reality visual analysis of intelligent human movement simulations built in virtual spaces can become part of real space. This research introduces a fundamentally new way to view and link simulations of people with the real-world context of the built environment: mixed reality crowd simulation in real space.

Advancements in extended reality (XR) have inspired new uses and users of advanced visualization interfaces, transforming geospatial data visualization and consumption by enabling interactive 3D geospatial data experiences in 3D. Conventional metrics (e.g., mental rotations test (MRT)) are often used to assess and predict the appropriateness of these visualizations without accounting for the effect the interface has on those metrics. We developed the Immersive MRT (IMRT) to evaluate the impact that virtual reality (VR) based visualizations and 3D virtual environments have on mental rotation performance. Consistent with previous work, the results of our pilot study suggest that mental rotation tasks are performed more accurately and rapidly with stereo 3D stimuli than with 2D images of those stimuli.