Culture is a field that is currently entering a revolutionary phase, no longer being a privilege for the few, but expanding to new audiences who are urged to not only passively consume cultural heritage content, but actually participate and assimilate it on their own. In this context, museums have already embraced new technologies as part of their exhibitions, many of them featuring augmented or virtual reality artifacts. The presented work proposes the synthesis of augmented, virtual and mixed reality technologies to provide unified X-Reality experiences in realistic virtual museums, engaging visitors in an interactive and seamless fusion of physical and virtual worlds that will feature virtual agents exhibiting naturalistic behavior. Visitors will be able to interact with the virtual agents, as they would with real world counterparts. The envisioned approach is expected to not only provide refined experiences for museum visitors, but also achieve high quality entertainment combined with more effective knowledge acquisition.

In this paper, a gaze-based Relevance Feedback (RF) approach to region-based image retrieval is presented. Fundamental idea of the proposed method comprises the iterative estimation of the real-world objects (or their constituent parts) that are of interest to the user and the subsequent exploitation of this information for refining the image retrieval results. Primary novelties of this work are: a) the introduction of a new set of gaze features for realizing user's relevance assessment prediction at region-level, and b) the design of a time-efficient and effective object-based RF framework for image retrieval. Regarding the interpretation of the gaze signal, a novel set of features is introduced by formalizing the problem under a mathematical perspective, contrary to the exclusive use of explicitly defined features that are in principle derived from the psychology domain. Apart from the temporal attributes, the proposed features also represent the spatial characteristics of the gaze signal, which have not been extensively studied in the literature so far. On the other hand, the developed object-based RF mechanism aims at overcoming the main limitation of region-based RF approaches, i.e., the frequently inaccurate estimation of the regions of interest in the retrieved images. Moreover, the incorporation of a single-camera image processing-based gaze tracker makes the overall system cost efficient and portable. As it is shown by the experimental evaluation, the proposed method outperforms representative global- and region-based explicit RF approaches, using a challenging general-purpose image dataset.

Background: Augmented reality (AR) and artificial intelligence (AI) are highly disruptive technologies that have revolutionised practices in a wide range of domains, including the security sector. Several law enforcement agencies (LEAs) employ AI in their daily operations for forensics and surveillance. AR is also gaining traction in security, particularly with the advent of affordable wearable devices. Equipping police officers with the tools to facilitate an elevated situational awareness (SA) in patrolling and tactical scenarios is expected to improve LEAs’ safety and capacity to deliver crucial blows against terrorist and/or criminal threats. Methods: In this paper we present DARLENE, an ecosystem incorporating novel AI techniques for activity recognition and pose estimation tasks, combined with a wearable AR framework for visualization of the inferenced results via dynamic content adaptation according to the wearer’s stress level and operational context. The concept has been validated with end-users through co-creation workshops, while the decision-making mechanism for enhancing LEAs’ SA has been assessed with experts. Regarding computer vision components, preliminary tests of the instance segmentation method for humans’ and objects’ detection have been conducted on a subset of videos from the RWF-2000 dataset for violence detection, which have also been used to test a human pose estimation method that has so far exhibited impressive results, constituting the basis of further developments in DARLENE. Results: Evaluation results highlight that target users are positive towards the adoption of the proposed solution in field operations, and that the SA decision-making mechanism produces highly acceptable outcomes. Evaluation of the computer vision components yielded promising results and identified opportunities for improvement. Conclusions: This work provides the context of the DARLENE ecosystem and presents the DARLENE architecture, analyses its individual technologies, and demonstrates preliminary results, which are positive both in terms of technological achievements and user acceptance of the proposed solution.

Real-time performance is critical for Augmented Reality (AR) systems as it directly affects responsiveness and enables the timely rendering of virtual content superimposed on real scenes. In this context, we present the DARLENE wearable AR system, analysing its specifications, overall architecture and core algorithmic components. DARLENE comprises AR glasses and a wearable computing node responsible for several time-critical computation tasks. These include computer vision modules developed for the real-time analysis of dynamic scenes supporting functionalities for instance segmentation, tracking and pose estimation. To meet real-time requirements in limited resources, concrete algorithmic adaptations and design choices are introduced. The proposed system further supports real-time video streaming and interconnection with external IoT nodes. To improve user experience, a novel approach is proposed for the adaptive rendering of AR content by considering the user’s stress level, the context of use and the environmental conditions for adjusting the level of presented information towards enhancing their situational awareness. Through extensive experiments, we evaluate the performance of individual components and end-to-end pipelines. As the proposed system targets time-critical security applications where it can be used to enhance police officers’ situational awareness, further experimental results involving end users are reported with respect to overall user experience, workload and evaluation of situational awareness.

Background: Patients undergoing surgery often experience stress and anxiety, which can increase complications and hinder recovery. Effective management of these psychological factors is key to improving outcomes. Preoperative anxiety is inversely correlated with the amount of information patients receive, but accessible, personalized support remains limited, especially in preoperative settings. Face-to-face education is often impractical due to resource constraints. Digital health (DH) interventions offer a promising alternative, enhancing patient engagement and empowerment. However, most current tools focus on providing information, overlooking the importance of personalization and psychological support. Objective: This study aimed to assess the viability of a DH intervention known as the Adhera CARINAE DH Program. This program is specifically designed to offer evidence-based and personalized stress- and anxiety-management techniques. It achieves this by using a comprehensive digital ecosystem that incorporates wearable devices, mobile apps, and virtual reality technologies. The intervention program also makes use of advanced data-driven techniques to deliver tailored patient education and lifestyle support. Methods: A total of 74 patients scheduled for surgery across 4 hospitals in 3 European countries were enrolled in this study from September 2021 to March 2022. Surgeries included cardiopulmonary and coronary artery bypass surgeries, cardiac valve replacements, prostate or bladder cancer surgeries, hip and knee replacements, maxillofacial surgery, and scoliosis procedures. After assessment for eligibility, participants were randomized into 2 groups: the intervention group (n=23) received the Adhera CARINAE DH intervention in addition to standard care, while the control group (n=27) received standard care alone. Psychological metrics such as self-efficacy, self-management, and mental well-being were assessed before and after the intervention, alongside physiological markers of stress. Results: The intervention group demonstrated significant improvements across several psychological outcomes. For example, Visual Analogue Scale Stress at the hospital improved at admission by 5% and at hospital discharge by 11.1% and Visual Analogue Scale Pain at admission improved by 31.2%. In addition, Hospital Anxiety and Depression Scale Anxiety after surgery improved by 15.6%, and Positive and Negative Affect Scale-Negative at hospital admission improved by 17.5%. Overall, patients in the intervention study spent 17.12% less days in the hospital. Besides these individual scores, the intervention group shows more positive relationships among the psychological dimensions of self-efficacy, self-management, and mental well-being, suggesting that the CARINAE solution could have a positive effect and impact on the reduction of stress and negative emotions. Conclusions: Our results provide an important first step toward a deeper understanding of optimizing DH solutions to support patients undergoing surgery and for potential applications in remote patient monitoring and communication. Trial Registration: ClinicalTrials.gov NCT05184725; https://clinicaltrials.gov/study/NCT05184725 International Registered Report Identifier (IRRID): RR2-10.2196/38536