Two-dimensional ultrasound (US) is widely used in minimally invasive cardiac procedures due to its convenience of use and noninvasive nature. However, the low quality of US images often limits their utility as a means for guiding procedures, since it is often difficult to relate the images to their anatomical context. To improve the interpretability of the US images while maintaining US as a flexible anatomical and functional real-time imaging modality, we describe a multimodality image navigation system that integrates 2D US images with their 3D context by registering them to high quality preoperative models based on magnetic resonance imaging (MRI) or computed tomography (CT) images. The mapping from such a model to the patient is completed using spatial and temporal registrations. Spatial registration is performed by a two-step rapid registration method that first approximately aligns the two images as a starting point to an automatic registration procedure. Temporal alignment is performed with the aid of electrocardiograph (ECG) signals and a latency compensation method. Registration accuracy is measured by calculating the TRE. Results show that the error between the US and preoperative images of a beating heart phantom is 1.7plusmn0.4 mm, with a similar performance being observed in in vivo animal experiments.

Objective: Emerging off-pump beating heart valve repair techniques offer patients less invasive alternatives for mitral valve (MV) repair. However, most of these techniques rely on the limited spatial and temporal resolution of transesophageal echocardiography (TEE) alone, which can make tool visualization and guidance challenging. Methods: Using a magnetic tracking system and integrated sensors, we created an augmented reality (AR) environment displaying virtual representations of important intracardiac landmarks registered to biplane TEE imaging. In a porcine model, we evaluated the AR guidance system versus TEE alone using the transapically delivered NeoChord DS1000 system to perform MV repair with chordal reconstruction. Results: Successful tool navigation from left ventricular apex to MV leaflet was achieved in 12 of 12 and 9 of 12 (P = 0.2) attempts with AR imaging and TEE alone, respectively. The distance errors of the tracked tool tip from the intended midline trajectory (5.2 ± 2.4 mm vs 16.8 ± 10.9 mm, P = 0.003), navigation times (16.7 ± 8.0 seconds vs 92.0 ± 84.5 seconds, P = 0.004), and total path lengths (225.2 ± 120.3 mm vs 1128.9 ± 931.1 mm, P = 0.003) were significantly shorter in the AR-guided trials compared with navigation with TEE alone. Furthermore, the potential for injury to other intracardiac structures was nearly 40-fold lower when using the AR imaging for tool navigation. The AR guidance also seemed to shorten the learning curve for novice surgeons. Conclusions: Augmented reality–enhanced TEE facilitates more direct and safe intracardiac navigation of the NeoChord DS tool from left ventricular apex to MV leaflet. Tracked tool path results demonstrate fourfold improved accuracy, fivefold shorter navigation times, and overall improved safety with AR imaging guidance.

Anesthetic nerve blocks are a common therapy performed in hospitals around the world to alleviate acute and chronic pain. Tracking systems have shown considerable promise in other forms of therapy, but little has been done to apply this technology in the field of anesthesia. We are developing a guidance system for combining tracked needles with non-invasive ultrasound (US) and patient-specific geometric models. In experiments with phantoms two augmented reality (AR) guidance systems were compared to the exclusive use of US for lumbar facet injection therapy. Anesthetists and anesthesia residents were able to place needles within 0.57mm of the intended targets using our AR systems compared to 5.77 mm using US alone. A preliminary cadaver study demonstrated the system was able to accurately place radio opaque dye on targets. The combination of real time US with tracked tools and AR guidance has the potential to replace CT and fluoroscopic guidance, thus reducing radiation dose to patients and clinicians, as well as reducing health care costs.

Objective: Transcatheter aortic valve implantation (TAVI) relies on fluoroscopy and nephrotoxic contrast medium for valve deployment. We propose an alternative guidance system using augmented reality (AR) and transesophageal echocardiography (TEE) to guide TAVI deployment. The goals of this study were to determine how consistently the aortic valve annulus is defined from TEE using different aortic valve landmarks and to compare AR guidance with fluoroscopic guidance of TAVI deployment in an aortic root model. Methods: Magnetic tracking sensors were integrated into the TAVI catheter and TEE probe, allowing these tools to be displayed in an AR environment. Variability in identifying aortic valve commissures and cuspal nadirs was assessed using TEE aortic root images. To compare AR guidance of TAVI deployment with fluoroscopic guidance, a TAVI stent was deployed 10 times in the aortic root model using each of the two guidance systems. Results: Commissures and nadirs were both investigated as features for defining the valve annulus in the AR guidance system. The commissures were identified more consistently than the nadirs, with intraobserver variability of 2.2 and 3.8 mm, respectively, and interobserver variability of 3.3 and 4.7 mm, respectively. The precision of TAVI deployment using fluoroscopic guidance was 3.4 mm, whereas the precision of AR guidance was 2.9 mm, and its overall accuracy was 3.4 mm. This indicates that both have similar performance. Conclusions: Aortic valve commissures can be identified more reliably than cuspal nadirs from TEE. The AR guidance system achieved similar deployment accuracy to that of fluoroscopy while eliminating the use and consequences of nephrotoxic contrast and radiation.