Transcranial Magnetic Stimulation (TMS) is an effective non-invasive treatment method for major depressive disorder. Accurate placement of an electromagnetic coil on the patient’s head during repetitive TMS is the key for stimulation of the desired brain regions and positive treatment outcome. Neuronavigation systems constitute the state-of-the-art method to accurately stimulate the appropriate brain region. Local separation of navigation information and the patient anatomy in combination with intricate visualisations and cumbersome setup limits the benefits and usability of this method. The present study addresses these problems by proposing an audiovisual Augmented reality (AR) system for coil positioning during TMS. The system sonifies and visualises translational and rotational differences between a target and the current instrument position using a minimalistic graphical user interface and auditory display. Effects of cross-modal integration on usability and targeting precision were shown in an experiment comparing audiovisual AR, audio AR and visual neuronavigation. Our approach revealed significant improvements in task time of all proposed AR conditions over neuronavigation (p < 0.001). Conversely, the neuronavigation system achieved significantly better targeting accuracy (p < 0.001). A purely auditory guidance achieved comparable performance as the audiovisual interface designs.
Screw placement in the correct angular trajectory is one of the most intricate tasks during spinal fusion surgery. Due to the crucial role of pedicle screw placement for the outcome of the operation, spinal navigation has been introduced into the clinical routine. Despite its positive effects on the precision and safety of the surgical procedure, local separation of the navigation information and the surgical site, combined with intricate visualizations, limit the benefits of the navigation systems. Instead of a tech-driven design, a focus on usability is required in new research approaches to enable advanced and effective visualizations. This work presents a new tool-mounted interface (TMI) for pedicle screw placement. By fixing a TMI onto the surgical instrument, physical de-coupling of the anatomical target and navigation information is resolved. A total of 18 surgeons participated in a usability study comparing the TMI to the state-of-the-art visualization on an external screen. With the usage of the TMI, significant improvements in system usability (Kruskal–Wallis test p < 0.05) were achieved. A significant reduction in mental demand and overall cognitive load, measured using a NASA-TLX (p < 0.05), were observed. Moreover, a general improvement in performance was shown by means of the surgical task time (one-way ANOVA p < 0.001).
About 20 percent of patients undergoing breast-conserving surgery require reoperation due to cancerous tissue remaining inside the breast. Breast cancer localization systems utilize auditory feedback to convey the distance between a localization probe and a small marker (seed) implanted into the breast tumor prior to surgery. However, no information on the location of the tumor margin is provided. To reduce the reoperation rate by improving the usability and accuracy of the surgical task, we developed an auditory display using shape sonification to assist with tumor margin localization. Accuracy and usability of the interactive shape sonification were determined on models of the female breast in three user studies with both breast surgeons and non-clinical participants. The comparative studies showed a significant increase in usability (p<0.05) and localization accuracy (p<0.001) of the shape sonification over the auditory feedback currently used in surgery.
Ophthalmic surgeries consist of highly demanding microsurgical tasks that require surgeons to sustain mental focus and extreme manual dexterity for extended periods. To improve surgical precision and decision-making, intraoperative Optical Coherence Tomography (iOCT) has been integrated into surgical microscopes, offering cross-sectional imaging of anatomical tissues and instruments alongside the traditional microscopic view. While the clinical benefits of iOCT have been demonstrated across various ophthalmic procedures, the impact of this auxiliary information on the surgeon’s cognitive load has not yet been explored. This work is the first study to investigate physiological and subjective cognitive load during simulated iOCT-guided retinal surgery in 17 novice users. The eye tracking capabilities of a virtual reality eye surgery simulator were used to capture blink rate and pupil diameter. Electrocardiography was used to record heart rate (HR) and heart rate variability (HRV). iOCT-guided surgery resulted in significantly higher cognitive load than traditional microscope-guided surgery, indicated by increased HR (p<0.01), HRV (p<0.05), subjective task load (p<0.001) (NASA-TLX, mental effort), and task time (p<0.001). However, our findings indicate that although iOCT initially imposed a higher cognitive load on novice users, it ultimately enhanced their depth-related targeting precision, which is crucial for improving surgical accuracy and overall outcomes in vitreoretinal surgery.
This work aims to leverage medical augmented reality (AR) technology to counter the shortage of medical experts in low-resource environments. We present a complete and cross-platform proof-of-concept AR system that enables remote users to teach and train medical procedures without expensive medical equipment or external sensors. By seeing the 3D viewpoint and head movements of the teacher, the student can follow the teacher’s actions on the real patient. Alternatively, it is possible to stream the 3D view of the patient from the student to the teacher, allowing the teacher to guide the student during the remote session. A pilot study of our system shows that it is easy to transfer detailed instructions through this remote teaching system and that the interface is easily accessible and intuitive for users. We provide a performant pipeline that synchronizes, compresses, and streams sensor data through parallel efficiency.