In the last decade, the use of virtual reality (VR) technologies has been increasing, as head-mounted display devices, such as the Meta Quest series, have become commercially accessible to everyday consumers. There are plenty of software applications developed for VR, ranging from games to serious training. Despite the fast-growing emergence of novel and natural gesture-based interface components, traditional windows, icons, menus, and pointer user interfaces still remain popular in these implementations. Therefore, navigating and interacting within 3-dimensional environments can still be challenging in virtual reality. This body of work will serve as a reference to the literature survey on 3-dimensional user interfaces and their implementations, and categorize them to reiterate a set of design recommendations for implementing 3-dimensional user interfaces in virtual reality. While questions revolve around the need for a 3-dimensional user interface if a 2-dimensional user interface suffices, we cannot ignore the capabilities and benefits offered when utilising 3-dimensional space. Unlike other work, this work proposes a novel metaphor-guided quadrant model to delineate the 3-dimensional user interface problem in an end-to-end manner. To achieve this, over 30 unique articles selected using a meta-analyses methodology were reviewed. The result highlights the dimensional semantics of the inputs and dimensional outcomes of the virtual targets. Notable reiterated design recommendations would match devices to interaction and then to the tasks in 3-dimensional environments, reducing the degrees of freedom, imposing constraints when possible, and not implementing one single best interaction technique. The results of this survey will be beneficial for virtual reality simulation designers or researchers attempting to determine the appropriate model when developing user interfaces for virtual reality content.

Line drawing with colorization is a popular art format and tool for architectural illustration. The goal of this research is toward generating a high-quality and natural-looking colorization based on an architectural line drawing. This paper presents a new Generative Adversarial Network (GAN)-based method, named ArchGANs, including ArchColGAN and ArchShdGAN. ArchColGAN is a GAN-based line-feature-aware network for stylized colorization generation. ArchShdGAN is a lighting effects generation network, from which the building depiction in 3D can benefit. In particular, ArchColGAN is able to maintain the important line features and the correlation property of building parts as well as reduce the uneven colorization caused by sparse lines. Moreover, we proposed a color enhancement method to further improve ArchColGAN. Besides the single line drawing images, we also extend our method to handle line drawing image sequences and achieve rotation animation. Experiments and studies demonstrate the effectiveness and usefulness of our proposed method for colorization prototyping.

3D Structure-based localization aims to estimate the 6-DOF camera pose of a query image by means of feature matches against a 3D Structure-from-Motion (SfM) point cloud. For city-scale SfM point clouds with tens of millions of points, it becomes more and more difficult to disambiguate matches. Therefore a 3D Structure-based localization method, which can efficiently handle matches with very large outlier ratios, is needed. We propose a two-stage outlier filtering framework for city-scale localization that leverages both visibility and geometry intrinsics of SfM point clouds. Firstly, we propose a visibility-based outlier filter, which is based on a bipartite visibility graph, to filter outliers on a coarse level. Secondly, we apply a geometry-based outlier filter to generate a set of fine-grained matches with a novel data-driven geometrical constraint for efficient inlier evaluation. The proposed two-stage outlier filtering framework only relies on intrinsic information of a SfM point cloud. It is thus widely applicable to be embedded into existing localization approaches. The experimental results on two real-world datasets demonstrate the effectiveness of the proposed two-stage outlier filtering framework for city-scale localization.

Metaverse applications often require many new 3D point cloud models that are unlabeled and that have never been seen before; this limited information results in difficulties for data-driven model analyses. In this paper, we propose a novel data-driven 3D point cloud analysis network GN-CNN that is suitable for such scenarios. We tackle the difficulties with a few-shot learning (FSL) approach by proposing an unsupervised generative adversarial network GN-GAN to generate prior knowledge and perform warm start pre-training for GN-CNN. Furthermore, the 3D models in the Metaverse are mostly acquired with a focus on the models’ visual appearances instead of the exact positions. Thus, conceptually, we also propose to augment the information by unleashing and incorporating local variance information, which conveys the appearance of the model. This is realized by introducing a graph convolution-enhanced combined multilayer perceptron operation (CMLP), namely GCMLP, to capture the local geometric relationship as well as a local normal-aware GeoConv, namely GNConv. The GN-GAN adopts an encoder–decoder structure and the GCMLP is used as the core operation of the encoder. It can perform the reconstruction task. The GNConv is used as the convolution-like operation in GN-CNN. The classification performance of GN-CNN is evaluated on ModelNet10 with an overall accuracy of 95.9%. Its few-shot learning performance is evaluated on ModelNet40, when the training set size is reduced to 30%, the overall classification accuracy can reach 91.8%, which is 2.5% higher than Geo-CNN. Experiments show that the proposed method could improve the accuracy in 3D point cloud classification tasks and under few-shot learning scenarios, compared with existing methods such as PointNet, PointNet++, DGCNN, and Geo-CNN, making it a beneficial method for Metaverse applications.

Patient decontamination is the act of removing or neutralising hazardous substances from an affected individual. To ensure adequate emergency preparedness, regulations require hospitals to train personnel in decontamination procedures regularly. To supplement in-person training, a virtual reality (VR) system is being developed for the training of hospital staff members in the mass decontamination of hazardous and toxic materials (HAZMAT) and/or radioactively contaminated casualties. As a demonstration of the concept, a primary VR prototype is designed to help users familiarize themselves with a chemical scanner tool, intended for examining victims for residual chemical hazards. This initial prototype showcases the benefits of using VR to create training simulations, complementing existing decontamination training methods in a secure and cost-effective manner. The proposed approach features a modularized user-centric design, a novel scanning simulation, and a high-realism virtual environment and workflow to enhance training effectiveness. A pilot user study and assessment suggest that new users were able to achieve a significant level of competency with VR, compared to users who underwent traditional training.