This manuscript identifies and documents unsolved problems and research challenges in the extended reality (XR) domain (i.e., virtual (VR), augmented (AR), and mixed reality (MR)). The manuscript is structured to include technology, design, and human factor perspectives. The text is visualization/display-focused, that is, other modalities such as audio, haptic, smell, and touch, while important for XR, are beyond the scope of this paper. We further narrow our focus to mainly geospatial research, with necessary deviations to other domains where these technologies are widely researched. The main objective of the study is to provide an overview of broader research challenges and directions in XR, especially in spatial sciences. Aside from the research challenges identified based on a comprehensive literature review, we provide case studies with original results from our own studies in each section as examples to demonstrate the relevance of the challenges in the current research. We believe that this paper will be of relevance to anyone who has scientific interest in extended reality, and/or uses these systems in their research.

Neural Style Transfer is a Computer Vision topic intending to transfer the visual appearance or the style of images to other images. Developments in deep learning nicely generate stylized images from texture-based examples or transfer the style of a photograph to another one. In map design, the style is a multi-dimensional complex problem related to recognizable visual salient features and topological arrangements, supporting the description of geographic spaces at a specific scale. The map style transfer is still at stake to generate a diversity of possible new styles to render geographical features. Generative adversarial Networks (GANs) techniques, well supporting image-to-image translation tasks, offer new perspectives for map style transfer. We propose to use accessible GAN architectures, in order to experiment and assess neural map style transfer to ortho-images, while using different map designs of various geographic spaces, from simple-styled (Plan maps) to complex-styled (old Cassini, Etat-Major, or Scan50 B&W). This transfer task and our global protocol are presented, including the sampling grid, the training and test of Pix2Pix and CycleGAN models, such as the perceptual assessment of the generated outputs. Promising results are discussed, opening research issues for neural map style transfer exploration with GANs.

Color palettes are widely used by artists to define colors of artworks and explore color designs. In general, artists select the colors of a palette by following a set of rules, e.g. contrast or relative luminance. Existing interactive palette exploration tools explore palette spaces following limited constraints defined as geometric configurations in color space e.g. harmony rules on the color wheel. Palette search algorithms sample palettes from color relations learned from an input dataset, however they cannot provide interactive user edits and palette refinement. We introduce in this work a new versatile formulation enabling the creation of constraint-based interactive palette exploration systems. Our technical contribution is a graph-based palette representation, from which we define palette exploration as a minimization problem that can be solved efficiently and provide real-time feedback. Based on our formulation, we introduce two interactive palette exploration strategies: constrained palette exploration, and for the first time, constrained palette interpolation. We demonstrate the performances of our approach on various application cases and evaluate how it helps users finding trade-offs between concurrent constraints.

The process of reading a topographic map requires users to recognize and learn the cartographic symbols of the key (or legend) while interpreting the territory as depicted on the map at a given level of abstraction (the form and nature of features, their saliency and relationships). We present the results of an empirical user study that aims to identify and assess the main graphical characteristics that are used by map users to recognize the design principles which constitute the topographic mapping style adopted by IGN (Institut National de l'Information Géographique et Forestière), France. Our results suggest that 91% of the participants were able to recognize an IGN-France topographic map amongst other topographic map products. We also determine which graphical characteristics play a role in the recognition of this cartographic style, either by visual memory or by visual perception, and identify the representation of relief, including contour lines and shaded relief, as one of the major graphical characteristics of the topographic mapping style of IGN-France. Moreover, the participants of our study considered the representation of touristic POI (points of interest), toponymy, typography, the main roads network, the individual buildings and the forests, to be essential for stylistic recognition.