Three decades after the first set of work on Medical Augmented Reality (MAR) was presented to the international community, and ten years after the deployment of the first MAR solutions into operating rooms, its exact definition, basic components, systematic design, and validation still lack a detailed discussion. This paper defines the basic components of any Augmented Reality (AR) solution and extends them to exemplary Medical Augmented Reality Systems (MARS). We use some of the original MARS applications developed at the Chair for Computer Aided Medical Procedures and deployed into medical schools for teaching anatomy and into operating rooms for telemedicine and surgical guidance throughout the last decades to identify the corresponding basic components. In this regard, the paper is not discussing all past or existing solutions but only aims at defining the principle components and discussing the particular domain modeling for MAR and its design-development-validation process, and providing exemplary cases through the past in-house developments of such solutions.

Background: Analysis of surgical instrument motion is applicable in surgical skill assessment and monitoring of the learning progress in laparoscopy. Current commercial instrument tracking technology (optical or electromagnetic) has specific limitations and is expensive. Therefore, in this study, we apply inexpensive, off-the-shelf inertial sensors to track laparoscopic instruments in a training scenario. Methods: We calibrated two laparoscopic instruments to the inertial sensor and investigated its accuracy on a 3D-printed phantom. In a user study during a one-week laparoscopy training course with medical students and physicians, we then documented and compared the training effect in laparoscopic tasks on a commercially available laparoscopy trainer (Laparo Analytic, Laparo Medical Simulators, Wilcza, Poland) and the newly developed tracking setup. Results: Eighteen participants (twelve medical students and six physicians) participated in the study. The student subgroup showed significantly poorer results for the count of swings (CS) and count of rotations (CR) at the beginning of the training compared to the physician subgroup (p = 0.012 and p = 0.042). After training, the student subgroup showed significant improvements in the rotatory angle sum, CS, and CR (p = 0.025, p = 0.004 and p = 0.024). After training, there were no significant differences between medical students and physicians. There was a strong correlation between the measured learning success (LS) from the data of our inertial measurement unit system (LS_IMU) and the Laparo Analytic (LS_Lap) (Pearson’s r = 0.79). Conclusion: In the current study, we observed a good and valid performance of inertial measurement units as a possible tool for instrument tracking and surgical skill assessment. Moreover, we conclude that the sensor can meaningfully examine the learning progress of medical students in an ex-vivo setting.

Purpose: Only a few studies have evaluated Augmented Reality (AR) in in vivo simulations compared to traditional laparoscopy; further research is especially needed regarding the most effective AR visualization technique. This pilot study aims to determine, under controlled conditions on a 3D-printed phantom, whether an AR laparoscope improves surgical outcomes over conventional laparoscopy without augmentation.Methods: We selected six surgical residents at a similar level of training and had them perform a laparoscopic task. The participants repeated the experiment three times, using different 3D phantoms and visualizations:Floating AR,Occlusion AR, and without any AR visualization (Control). Surgical performance was determined using objective measurements. Subjective measures, such as task load and potential application areas, were collected with questionnaires.Results: Differences in operative time, total touching time, and SurgTLX scores showed no statistical significance ( $$p>0.05$$p>0.05 ). However, when assessing the invasiveness of the simulated intervention, the comparison revealed a statistically significant difference ( $$p=0.009$$p=0.009 ). Participants felt AR could be useful for various surgeries, especially for liver, sigmoid, and pancreatic resections (100%). Almost all participants agreed that AR could potentially lead to improved surgical parameters, such as operative time (83%), complication rate (83%), and identifying risk structures (83%).Conclusion: According to our results, AR may have great potential in visceral surgery and based on the objective measures of the study, may improve surgeons' performance in terms of an atraumatic approach. In this pilot study, participants consistently took more time to complete the task, had more contact with the vascular tree, were significantly more invasive, and scored higher on the SurgTLX survey than with AR.

Introduction: This study compares five augmented reality (AR) vasculature visualization techniques in a mixed-reality laparoscopy simulator with 50 medical professionals and analyzes their impact on the surgeon. Material and methods: The different visualization techniques' abilities to convey depth were measured using the participant's accuracy in an objective depth sorting task. Demographic data and subjective measures, such as the preference of each AR visualization technique and potential application areas, were collected with questionnaires. Results: Despite measuring differences in objective measurements across the visualization techniques, they were not statistically significant. In the subjective measures, however, 55% of the participants rated visualization technique II, 'Opaque with single-color Fresnel highlights', as their favorite. Participants felt that AR could be useful for various surgeries, especially complex surgeries (100%). Almost all participants agreed that AR could potentially improve surgical parameters, such as patient safety (88%), complication rate (84%), and identifying risk structures (96%). Conclusions: More studies are needed on the effect of different visualizations on task performance, as well as more sophisticated and effective visualization techniques for the operating room. With the findings of this study, we encourage the development of new study setups to advance surgical AR.