5 September 2019 in IEEE Transactions on Medical Imaging
Cardiac interventional procedures are often performed under fluoroscopic guidance, exposing both the patient and operators to ionizing radiation. To reduce this risk of radiation exposure, we are exploring the use of photoacoustic imaging paired with robotic visual servoing for cardiac catheter visualization and surgical guidance. A cardiac catheterization procedure was performed on two in vivo swine after inserting an optical fiber into the cardiac catheter to produce photoacoustic signals from the tip of the fiber-catheter pair. A combination of photoacoustic imaging and robotic visual servoing was employed to visualize and maintain constant sight of the catheter tip in order to guide the catheter through the femoral or jugular vein, toward the heart. Fluoroscopy provided initial ground truth estimates for 1D validation of the catheter tip positions, and these estimates were refined using a 3D electromagnetic-based cardiac mapping system as the ground truth. The 1D and 3D root mean square errors ranged 0.25-2.28 mm and 1.24-1.54 mm, respectively. The catheter tip was additionally visualized at three locations within the heart: (1) inside the right atrium, (2) in contact with the right ventricular outflow tract, and (3) inside the right ventricle. Lasered regions of cardiac tissue were resected for histopathological analysis, which revealed no laserrelated tissue damage, despite the use of 2.98 mJ per pulse at the fiber tip (379.2 mJ/cm2 fluence). In addition, there was a 19 dB difference in photoacoustic signal contrast when visualizing the catheter tip pre-and post-endocardial tissue contact, which is promising for contact confirmation during cardiac interventional procedures (e.g., cardiac radiofrequency ablation). These results are additionally promising for the use of photoacoustic imaging to guide cardiac interventions by providing depth information and enhanced visualization of catheter tip locations within blood vessels and within the beating heart.
21 April 2022 in IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control
The successful integration of computer vision, robotic actuation, and photoacoustic imaging to find and follow targets of interest during surgical and interventional procedures requires accurate photoacoustic target detectability. This detectability has traditionally been assessed with image quality metrics, such as contrast, contrast-to-noise ratio, and signal-to-noise ratio (SNR). However, predicting target tracking performance expectations when using these traditional metrics is difficult due to unbounded values and sensitivity to image manipulation techniques like thresholding. The generalized contrast-to-noise ratio (gCNR) is a recently introduced alternative target detectability metric, with previous work dedicated to empirical demonstrations of applicability to photoacoustic images. In this article, we present theoretical approaches to model and predict the gCNR of photoacoustic images with an associated theoretical framework to analyze relationships between imaging system parameters and computer vision task performance. Our theoretical gCNR predictions are validated with histogram-based gCNR measurements from simulated, experimental phantom, ex vivo , and in vivo datasets. The mean absolute errors between predicted and measured gCNR values ranged from ${3.2} \times {10}^{-{3}}$ to ${2.3} \times {10}^{-{2}}$ for each dataset, with channel SNRs ranging −40 to 40 dB and laser energies ranging 0.07 $\mu \text{J}$ to 68 mJ. Relationships among gCNR, laser energy, target and background image parameters, target segmentation, and threshold levels were also investigated. Results provide a promising foundation to enable predictions of photoacoustic gCNR and visual servoing segmentation accuracy. The efficiency of precursory surgical and interventional tasks (e.g., energy selection for photoacoustic-guided surgeries) may also be improved with the proposed framework.
13 November 2023 in Journal of Biomedical Optics
Cardiac interventional procedures are often performed to diagnose and treat cardiac arrhythmias (e.g., 18,000 to 44,500 cardiac catheter ablation procedures have been performed annually in the United States1). These procedures generally require catheter delivery from an insertion point in the thigh to the heart via the femoral vein. One of the most serious and potentially life-threatening complications of catheter ablations is the risk of cardiac perforation,1 which can be minimized with state-of-the-art catheter tip visualization methods.
1 January 2026 in IEEE Transactions on Biomedical Engineering
Objective: To develop the first known deep learning-based photoacoustic visual servoing system utilizing point source localization and hybrid position-force control to track catheter tips in three dimensions in real-time. Methods: We integrated either object detection or instance segmentation-based localization with hybrid position-force control to create our novel system. Cardiac catheter tips were then tracked across distances of 40 mm in a plastisol phantom and 25-64 mm in an in vivo swine in real-time in nine visual servoing trials total. Results: Object detection-based localization identified the cardiac catheter tip in 88.0-91.7% and 66.7-70.4% of phantom and in vivo channel data frames, respectively. Instance segmentation detection rates ranged 86.4-100.0% in vivo. These catheter tips were tracked with errors as low as 0.5 mm in phantom trials and 0.8 mm in the in vivo trials. The mean inference times were ≥145.3 ms and ≥516.3 ms with object detection-based and instance segmentation-based point source localization, respectively. Hybrid position-force control system enabled contact with the imaging surface during ≥99.43% of each visual servoing trial. Conclusion: Our novel deep learning-based photoacoustic visual servoing system was successfully demonstrated. Object detectionbased localization operated with inference times that are more suitable for real-time implementations while instance segmentation had lower tracking errors. Significance: After implementing suggested optimization modifications, our novel system has the potential to track catheter tips, needle tips, and other surgical tool tips in real-time during surgical and interventional procedures.
14 November 2024 in Biophotonics Discovery
SignificancePhotoacoustic imaging has the potential to improve non-invasive breast cancer diagnosis. However, illumination through the skin introduces a skin tone bias, as greater melanin content increases optical absorption and can create acoustic clutter, reducing the visibility of various target sizes.AimWe investigate the impact of skin tone bias as a function of target sizes in three photoacoustic image reconstruction methods: fast Fourier transform (FFT)-based reconstruction, delay-and-sum (DAS) beamforming, and short-lag spatial coherence (SLSC) beamforming.MethodsThe three imaging methods were applied to channel data from multidomain simulations with 757, 800, and 1064 nm wavelengths, 11 target sizes (ranging 0.5 to 3 mm), 18 skin tones [ranging individual typology angles (ITAs), of –54 deg to 60 deg], and a previously validated realistic 3D breast model. The signal-to-noise ratio (SNR) and generalized contrast-to-noise ratio (gCNR) were measured to assess visibility.ResultsWith 757 and 800 nm wavelengths, targets underlying dark skin tones (ITA range: −54 deg to −33 deg) with sizes ≤3 mm were poorly visible with ≤2.38 to 4.21 mean SNR and ≤0.46 to 0.74 mean gCNR, with smaller targets generally being more visible with lighter skin tones. A 1064-nm transmit wavelength improved visualization with DAS and SLSC beamforming, relative to both FFT reconstruction with a 1064 nm wavelength and DAS or SLSC beamforming with 757 and 800 nm wavelengths. When combined with SLSC beamforming, the 1064-nm wavelength offered the greatest improvements, enabling visualization of simulated target sizes ranging from 0.5 to 3 mm underlying very light (ITA = 60 deg) to dark (ITA=−54 deg) skin tones, with mean SNR ≤10.01 and mean gCNR ≤1. Visualization of simulated vessel structures derived from in vivo photoacoustic images was consistent with simulation-based target size expectations.ConclusionsResults are promising for advancing next-generation photoacoustic imaging systems for breast cancer diagnosis across the range of skin tones represented in healthcare systems throughout the world.
22 April 2025 in IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control
Surgical tool tip localization and tracking are essential components of surgical and interventional procedures. The cross sections of tool tips can be considered as acoustic point sources to achieve these tasks with deep learning applied to photoacoustic channel data. However, source localization was previously limited to the lateral and axial dimensions of an ultrasound transducer. In this paper, we developed a novel deep learning-based three-dimensional (3D) photoacoustic point source localization system using an object detection-based approach extended from our previous work. In addition, we derived theoretical relationships among point source locations, sound speeds, and waveform shapes in raw photoacoustic channel data frames. We then used this theory to develop a novel deep learning instance segmentation-based 3D point source localization system. When tested with 4,000 simulated, 993 phantom, and 1,983 ex vivo channel data frames, the two systems achieved F1 scores as high as 99.82%, 93.05%, and 98.20%, respectively, and Euclidean localization errors (mean ± one standard deviation) as low as 1.46±1.11 mm, 1.58±1.30 mm, and 1.55±0.86 mm, respectively. In addition, the instance segmentation-based system simultaneously estimated sound speeds with absolute errors (mean ± one standard deviation) of 19.22±26.26 m/s in simulated data and standard deviations ranging 14.6-32.3 m/s in experimental data. These results demonstrate the potential of the proposed photoacoustic imaging-based methods to localize and track tool tips in three dimensions during surgical and interventional procedures.