Major hurdles for Microsoft's HoloLens as a tool in medicine have been accessing tracking data, as well as a relatively high‐localisation error of the displayed information; cumulatively resulting in its limited use and minimal quantification. The following work investigates the augmentation of HoloLens with the proprietary image processing SDK Vuforia, allowing integration of data from its front‐facing RGB camera to provide more spatially stable holograms for neuronavigational use. Continuous camera tracking was able to maintain hologram registration with a mean perceived drift of 1.41 mm, as well as a mean sub 2‐mm surface point localisation accuracy of 53%, all while allowing the researcher to walk about a test area. This represents a 68% improvement for the later and a 34% improvement for the former compared with a typical HoloLens deployment used as a control. Both represent a significant improvement on hologram stability given the current state‐of‐the‐art, and to the best of the authors knowledge are the first example of quantified measurements when augmenting hologram stability using data from the RGB sensor.

The integration of Deep Learning (DL) models with the HoloLens2 Augmented Reality (AR) headset has enormous potential for real-time AR medical applications. Currently, most applications execute the models on an external server that communicates with the headset via Wi-Fi. This client-server architecture introduces undesirable delays and lacks reliability for real-time applications. However, due to HoloLens2’s limited computation capabilities, running the DL model directly on the device and achieving real-time performances is not trivial. Therefore, this study has two primary objectives: (i) to systematically evaluate two popular frameworks to execute DL models on HoloLens2—Unity Barracuda and Windows Machine Learning (WinML)—using the inference time as the primary evaluation metric; (ii) to provide benchmark values for state-of-the-art DL models that can be integrated in different medical applications (e.g., Yolo and Unet models). In this study, we executed DL models with various complexities and analyzed inference times ranging from a few milliseconds to seconds. Our results show that Unity Barracuda is significantly faster than WinML (p-value < 0.005). With our findings, we sought to provide practical guidance and reference values for future studies aiming to develop single, portable AR systems for real-time medical assistance.

BACKGROUND AND OBJECTIVES: External ventricular drain (EVD) placement is often performed freehand, a technique subpar to accurate yet impractical image-guided methods, yielding optimal placement in only 70%. The aim of this study was to address shortcomings in EVD placement and image guidance technologies by implementing high-accuracy augmented reality (AR) guidance. METHODS: We conducted a prospective clinical pilot study to assess feasibility, safety, and clinical performance of EVD placement using a standalone AR headset equipped with high-accuracy inside-out infrared tracking and software addressing EVD placement. Placement quality was reported using a newly defined extended modified Kakarla scale, and dichotomized into clinically relevant outcome parameters. Results were compared with a nonconcurrent freehand control group using one-sided Fisher exact tests. RESULTS: Eleven AR-guided EVD placements were performed, achieving functional placement in all cases on the first attempt, vs 7 (64%) in the control group (P = .045); successful placement in 9 (82%) vs 5 (45%); optimal in 8 (73%) vs 3 (27%) (P = .043); suboptimal in 2 (18%) vs 5 (45%); and failed in 0 vs 1 (9%). No AR-guided placements required revision, whereas the freehand group had a 36% reintervention rate (P = .045). Procedure-related complications occurred in 2 AR-guided cases (18%), vs 5 (45%) freehand (all post-reintervention). CONCLUSION: This study presents the first clinical use case of EVD placement using high-accuracy AR guidance contained in a standalone head-worn navigation system. Safe and reliable outcomes using a validated pipeline were demonstrated, eliminating stick-and-poke attempts and resulting in improved quality, increased single attempt success rates, and reduced revision and complication rates. Based on these results, a multicenter randomized controlled trial will be initiated.

Purpose: Computer-aided navigation and patient-specific 3D printed guides have demonstrated superior outcomes in total shoulder arthroplasty (TSA). Nevertheless, few TSAs are inserted using these technologies. Head-worn augmented reality (AR) devices can provide intuitive 3D computer navigation to the surgeon. This study investigates AR navigation in conjunction with adaptive spatial drift correction toward TSA. Methods: A phantom study was performed to assess the performance of AR navigated pin placement in TSA. Two medical experts performed a total of 12 pin placements into phantom scapula; six were placed using an end-to-end AR-navigated technique, and six using a common freehand technique. Inside-out infrared (IR) tracking was designed and integrated into the AR headset to correct for device drift and provide tool tracking. Additionally, the impact of IR tool tracking, registration, and superposed/juxtaposed visualization techniques was investigated. Results: The AR-navigated pin placement resulted in a mean entry point error of 1.06 mm ± 0.64 mm and directional error of $${1.66^\circ \pm 0.65^\circ }$$ 1 . 66 ∘ ± 0 . 65 ∘ . Compared with the freehand technique, AR navigation resulted in improved directional outcomes ( $$p=0.03$$ p = 0.03 ), while entry point accuracy was not significantly different ( $$p=0.44$$ p = 0.44 ). IR tool tracking error was 1.47 mm ± 0.69 mm and $${0.92^\circ \pm 0.50^\circ }$$ 0 . 92 ∘ ± 0 . 50 ∘ , and registration error was 4.32 mm ± 1.75 mm and $${2.56^\circ \pm 0.82^\circ }$$ 2 . 56 ∘ ± 0 . 82 ∘ . No statistical difference between AR visualization techniques was found in entry point ( $$p=0.22$$ p = 0.22 ) or directional ( $$p=0.31$$ p = 0.31 ) errors. Conclusion: AR navigation allowed for comparable pin placement outcomes with those reported in the literature for patient-specific 3D printed guides; moreover, it complements the patient-specific planning without the need for the guides themselves.

Background: We present a method to determine and visualise the functional centre of rotation (FCOR) of the hip during total hip arthroplasty using an augmented reality head mounted display (AR‐HMD).Methods: We developed software allowing a HoloLens to provide inside‐out infrared tracking of markers affixed to cadaver femurs and 3D printed acetabuli. Two observers rotated 20 cadaver femurs twice in a matching cup, producing 80 measurements. The FCOR of the acetabulum and femoral head was determined based on the point cloud generated from the displacement of the femoral trackers to the acetabular tracker.Results: Compared to the ground truth, the FCOR resulted in an absolute error of 2.9 ± 1.4 mm for the acetabulum and 2.9 ± 1.2 mm for the femur, with 95th percentiles below 5.6 and 4.7 mm.Conclusion: The proposed AR‐HMD system offers an accurate and reproducible way to determine the femoral and acetabular FCOR in an experimental setting.