Culture is a field that is currently entering a revolutionary phase, no longer being a privilege for the few, but expanding to new audiences who are urged to not only passively consume cultural heritage content, but actually participate and assimilate it on their own. In this context, museums have already embraced new technologies as part of their exhibitions, many of them featuring augmented or virtual reality artifacts. The presented work proposes the synthesis of augmented, virtual and mixed reality technologies to provide unified X-Reality experiences in realistic virtual museums, engaging visitors in an interactive and seamless fusion of physical and virtual worlds that will feature virtual agents exhibiting naturalistic behavior. Visitors will be able to interact with the virtual agents, as they would with real world counterparts. The envisioned approach is expected to not only provide refined experiences for museum visitors, but also achieve high quality entertainment combined with more effective knowledge acquisition.

In this work, a novel computational approach for the dynamic adaptation of User Interfaces (UIs) is proposed, which aims at enhancing the Situational Awareness (SA) of users by leveraging the current context and providing the most useful information, in an optimal and efficient manner. By combining Ontology modeling and reasoning with Combinatorial Optimization, the system decides what information to present, when to present it, where to visualize it in the display - and how , taking into consideration contextual factors as well as placement constraints. The main objective of the proposed approach is to optimize the SA associated with the displayed UI at run-time , while avoiding information overload and induced stress. In the context of this work, we have deployed our computational approach to the use case of an Augmented Reality (AR) system for Law Enforcement Agents (LEAs). To explore the benefits and limitations of the developed system, two evaluations have been conducted. The first one was an expert-based evaluation with LEAs and User Experience (UX) experts, assessing the appropriateness of the system’s decisions. The second one was a user-based evaluation involving LEAs from different agencies, estimating the SA, the mental workload and the overall UX associated with the system, through an AR simulation. The results indicate that the system enhances SA, and while not imposing workload, it provides an overall positive UX.

‘User Experience’ (UX) is a term that has been established in HCI research and practice, subsuming the term ‘usability’. UX denotes that interaction with a contemporary technological system goes far beyond usability, extending to one’s emotions before, during, and after using the system and cannot be defined only by studying the fundamental usability attributes of effectiveness, efficiency and user satisfaction. Measuring UX becomes a substantially more complicated endeavor when the interaction target is not just a technological system or application, but an entire intelligent environment and the systems contained therein. Motivated by the imminent need to assess, measure and quantify user experience in intelligent environments, this paper presents a methodological and conceptual framework that provides concrete guidance for UX research, design and evaluation, explaining which UX parameter should be measured, how, and when. An evaluation of the framework indicated that it can be valuable for researchers and practitioners, assisting them in planning, carrying out, and analyzing UX studies in a comprehensive and thorough manner, thus enhancing their understanding and improving the experiences they design for intelligent environments.

Human stress is intricately linked with mental processes such as decision making. Public protection practitioners, including Law Enforcement Agents (LEAs), are forced to make difficult decisions during high-pressure operations, under strenuous circumstances. In this respect, systems and applications that assist such practitioners to take decisions, are increasingly incorporating user stress level information for their development, adaptation, and evaluation. To that end, our goal is to accurately detect and classify the level of acute, short-term stress, in real time, for the development of personalized, context-aware solutions for LEAs. Deep Neural Networks (DNNs), and in particular Convolutional Neural Networks (CNNs), have been gaining traction in the field of stress analysis, exhibiting promising results. Furthermore, the electrocardiogram (ECG) signals, have also been widely adopted for estimating levels of stress. In this work, we propose two CNN architectures for the stress detection and 3-level (low, moderate, high) stress classification tasks, using ultra short-term raw ECG signals (3 s). One architecture is simple and with a low memory footprint, suitable for running in wearable edge-computing nodes, and the other is able to learn more complex features, having more trainable parameters. The models were trained on the two publicly available stress classification datasets, after applying pre-processing techniques, such as data pruning, down-sampling, and data augmentation, using a sliding window approach. After hyperparameter tuning, using 4-fold cross-validation, the evaluation on the test set demonstrated state-of-the-art accuracy both on the 3- and 2-level stress classification task using the DriveDB dataset, reporting an accuracy of 83.55% and 98.77% respectively.

Background: Augmented reality (AR) and artificial intelligence (AI) are highly disruptive technologies that have revolutionised practices in a wide range of domains, including the security sector. Several law enforcement agencies (LEAs) employ AI in their daily operations for forensics and surveillance. AR is also gaining traction in security, particularly with the advent of affordable wearable devices. Equipping police officers with the tools to facilitate an elevated situational awareness (SA) in patrolling and tactical scenarios is expected to improve LEAs’ safety and capacity to deliver crucial blows against terrorist and/or criminal threats. Methods: In this paper we present DARLENE, an ecosystem incorporating novel AI techniques for activity recognition and pose estimation tasks, combined with a wearable AR framework for visualization of the inferenced results via dynamic content adaptation according to the wearer’s stress level and operational context. The concept has been validated with end-users through co-creation workshops, while the decision-making mechanism for enhancing LEAs’ SA has been assessed with experts. Regarding computer vision components, preliminary tests of the instance segmentation method for humans’ and objects’ detection have been conducted on a subset of videos from the RWF-2000 dataset for violence detection, which have also been used to test a human pose estimation method that has so far exhibited impressive results, constituting the basis of further developments in DARLENE. Results: Evaluation results highlight that target users are positive towards the adoption of the proposed solution in field operations, and that the SA decision-making mechanism produces highly acceptable outcomes. Evaluation of the computer vision components yielded promising results and identified opportunities for improvement. Conclusions: This work provides the context of the DARLENE ecosystem and presents the DARLENE architecture, analyses its individual technologies, and demonstrates preliminary results, which are positive both in terms of technological achievements and user acceptance of the proposed solution.