The integration of Augmented Reality (AR) technology into surgical procedures offers significant potential to enhance clinical outcomes. Despite numerous lab-proven prototypes, deploying these systems in actual clinical settings demands specialized design and rigorous clinical evaluation to meet the high standards of complex medical fields. Our research highlights the intricate requirements emerging from clinical environments, particularly operating theaters. To address these challenges, we introduce ARAS, an operational AR assistance system for open pancreatic surgery. Through a user-centric design methodology, ARAS was iteratively developed and refined, ensuring its practical applicability and effectiveness in real-world surgical settings during clinical trials. ARAS encompasses two different modes designed for preoperative and intraoperative sessions. Preoperative mode enables surgeons to visualize patient data and perform resection on the patient's 3D reconstructed vessels and tumors, helping surgeons during surgical planning. Intraoperative mode allows in-situ and precise visualization of these reconstructed 3D models during the surgical procedure and supports surgeons in decision-making. ARAS also includes a novel interaction design for surgical AR systems. It uses LLMs to enable context-aware, intuitive, and natural communication between surgeons and the AR system. This demo showcases the capabilities of ARAS, emphasizing its potential to transform surgical practices through advanced AR and LLM-supported interactions.

Background: The emergence of next-generation video-see-through head-mounted displays, such as the Apple Vision Pro (AVP), has generated considerable interest in the medical field. While preliminary studies highlight AVP’s potential, no controlled study has rigorously assessed its usability for precision-based medical tasks requiring fine motor control and real-world perception. Objective: This study aims to evaluate the feasibility of using AVP while performing real-world medical precision tasks. Methods: To assess AVP’s feasibility, we conducted a controlled user study with 20 health care professionals, who performed 3 different suturing techniques across 3 intervention conditions. Participants completed the same tasks using AVP, the Microsoft HoloLens 2 (MHL2), and a baseline (without a head-mounted display). A within-subject design was used, ensuring that each participant experienced all intervention groups. We used a mixed methods research approach, incorporating both quantitative metrics, including task completion time, suturing performance, system usability score, cognitive load, virtual reality sickness, and presence score, as well as qualitative insights gathered through interviews. Results: Participants took significantly longer to complete the entire task using AVP (570.0, SD 192.0 s) compared with MHL2 (456.0, SD 120.0 s; P<.001) and baseline (472.0, SD 143.0 s; P<.001). The analysis on participants’ average suture performance revealed no significant differences across interventions (P=.76). The total raw NASA Task Load Index score among participants was significantly higher for AVP (43.9, SD 15.9) compared with MHL2 (21.5, SD 13.8; P<.001) and baseline (19.1, SD 15.1; P<.001). The analysis of the presence questionnaire demonstrated a significantly higher presence score for MHL2 (115.0, SD 11.4) compared with AVP (93.7, SD 12.7; P<.001). The overall virtual reality sickness questionnaire score was significantly higher for AVP (66.9, SD 19.8) compared with MHL2 (41.1, SD 9.32; P<.001). Moreover, the calculated system usability score for MHL2 (72.7, SD 8.54) was significantly higher compared with AVP (50.3, SD 14.4; P<.001). Conclusions: In conclusion, AVP has potential for non–time-sensitive medical applications or those that emphasize digital elements over real-world interaction. Its current usability limitations, particularly increased cognitive load and prolonged task execution times, suggest that further optimizations are necessary before widespread clinical adoption is feasible.

This paper investigates the use of augmented reality (AR) technology to deliver augmented lectures to support students in acquiring the curricular competency of using spatial imagination in mathematics education. As a very important stage in education to develop spatial abilities this paper focuses on elementary school children. Due to the challenges of working with children in experimental studies, this education level has received comparatively little attention in terms of support through ubiquitous technology. A theory-driven design approach was adopted in the development of an AR-based learning environment to visualize various virtual 3D cube buildings aligned with real blueprints. The designed environment was evaluated by a group of 1st and 2nd grade students in terms of system usability. The results showed that the theory-driven design was successful with a score of 86.56 on the System Usability test. In our future work, we aimed to assess the effectiveness of the proposed AR environment in terms of learning gains by performing more task-focused studies before and after experiment and comparing the results with the traditional teaching methods.

The rising cycling trend in recent years has highlighted the need for enhanced safety, especially with increasing cyclist numbers and changing infrastructure dynamics. Fueled by the increasing smartphone usage, distracted riding due to digital devices remains a critical concern. In this work, we propose a system utilizing capacitive sensing for hands- and eyes-free interaction, designed to improve safety while maintaining digital connectivity. Our system employs capacitive shoulder covers that detect shoulder tap gestures with the user's cheeks, allowing cyclists to interact with their smartphones without removing their hands from the handlebars or diverting their gaze from the road. A mobile application deployed on Android forms the interface to detect gestures through a Convolutional Neural Network and link them with appropriate smartphone features. Potential use cases include phone call management, media player control, navigation confirmation, and broader integration towards cyclist communication and safety in swarm cycling scenarios.