Deep learning for ultrasound image formation is rapidly garnering research support and attention, quickly rising as the latest frontier in ultrasound image formation, with much promise to balance both image quality and display speed. Despite this promise, one challenge with identifying optimal solutions is the absence of unified evaluation methods and datasets that are not specific to a single research group. This article introduces the largest known international database of ultrasound channel data and describes the associated evaluation methods that were initially developed for the challenge on ultrasound beamforming with deep learning (CUBDL), which was offered as a component of the 2020 IEEE International Ultrasonics Symposium. We summarize the challenge results and present qualitative and quantitative assessments using both the initially closed CUBDL evaluation test dataset (which was crowd-sourced from multiple groups around the world) and additional in vivo breast ultrasound data contributed after the challenge was completed. As an example quantitative assessment, single plane wave images from the CUBDL Task 1 dataset produced a mean generalized contrast-to-noise ratio (gCNR) of 0.67 and a mean lateral resolution of 0.42 mm when formed with delay-and-sum beamforming, compared with a mean gCNR as high as 0.81 and a mean lateral resolution as low as 0.32 mm when formed with networks submitted by the challenge winners. We also describe contributed CUBDL data that may be used for training of future networks. The compiled database includes a total of 576 image acquisition sequences. We additionally introduce a neural-network-based global sound speed estimator implementation that was necessary to fairly evaluate the results obtained with this international database. The integration of CUBDL evaluation methods, evaluation code, network weights from the challenge winners, and all datasets described herein are publicly available (visit https://cubdl.jhu.edu for details).

Cardiac interventional procedures are often performed under fluoroscopic guidance, exposing both the patient and operators to ionizing radiation. To reduce this risk of radiation exposure, we are exploring the use of photoacoustic imaging paired with robotic visual servoing for cardiac catheter visualization and surgical guidance. A cardiac catheterization procedure was performed on two in vivo swine after inserting an optical fiber into the cardiac catheter to produce photoacoustic signals from the tip of the fiber-catheter pair. A combination of photoacoustic imaging and robotic visual servoing was employed to visualize and maintain constant sight of the catheter tip in order to guide the catheter through the femoral or jugular vein, toward the heart. Fluoroscopy provided initial ground truth estimates for 1D validation of the catheter tip positions, and these estimates were refined using a 3D electromagnetic-based cardiac mapping system as the ground truth. The 1D and 3D root mean square errors ranged 0.25-2.28 mm and 1.24-1.54 mm, respectively. The catheter tip was additionally visualized at three locations within the heart: (1) inside the right atrium, (2) in contact with the right ventricular outflow tract, and (3) inside the right ventricle. Lasered regions of cardiac tissue were resected for histopathological analysis, which revealed no laserrelated tissue damage, despite the use of 2.98 mJ per pulse at the fiber tip (379.2 mJ/cm2 fluence). In addition, there was a 19 dB difference in photoacoustic signal contrast when visualizing the catheter tip pre-and post-endocardial tissue contact, which is promising for contact confirmation during cardiac interventional procedures (e.g., cardiac radiofrequency ablation). These results are additionally promising for the use of photoacoustic imaging to guide cardiac interventions by providing depth information and enhanced visualization of catheter tip locations within blood vessels and within the beating heart.

Photoacoustic imaging–the combination of optics and acoustics to visualize differences in optical absorption – has recently demonstrated strong viability as a promising method to provide critical guidance of multiple surgeries and procedures. Benefits include its potential to assist with tumor resection, identify hemorrhaged and ablated tissue, visualize metal implants (e.g., needle tips, tool tips, brachytherapy seeds), track catheter tips, and avoid accidental injury to critical subsurface anatomy (e.g., major vessels and nerves hidden by tissue during surgery). These benefits are significant because they reduce surgical error, associated surgery-related complications (e.g., cancer recurrence, paralysis, excessive bleeding), and accidental patient death in the operating room. This invited review covers multiple aspects of the use of photoacoustic imaging to guide both surgical and related non-surgical interventions. Applicable organ systems span structures within the head to contents of the toes, with an eye toward surgical and interventional translation for the benefit of patients and for use in operating rooms and interventional suites worldwide. We additionally include a critical discussion of complete systems and tools needed to maximize the success of surgical and interventional applications of photoacoustic-based technology, spanning light delivery, acoustic detection, and robotic methods. Multiple enabling hardware and software integration components are also discussed, concluding with a summary and future outlook based on the current state of technological developments, recent achievements, and possible new directions.

Deep fully connected networks are often considered “universal approximators” that are capable of learning any function. In this article, we utilize this particular property of deep neural networks (DNNs) to estimate normalized cross correlation as a function of spatial lag (i.e., spatial coherence functions) for applications in coherence-based beamforming, specifically short-lag spatial coherence (SLSC) beamforming. We detail the composition, assess the performance, and evaluate the computational efficiency of CohereNet, our custom fully connected DNN, which was trained to estimate the spatial coherence functions of in vivo breast data from 18 unique patients. CohereNet performance was evaluated on in vivo breast data from three additional patients who were not included during training, as well as data from in vivo liver and tissue mimicking phantoms scanned with a variety of ultrasound transducer array geometries and two different ultrasound systems. The mean correlation between the SLSC images computed on a central processing unit (CPU) and the corresponding DNN SLSC images created with CohereNet was 0.93 across the entire test set. The DNN SLSC approach was up to 3.4 times faster than the CPU SLSC approach, with similar computational speed, less variability in computational times, and improved image quality compared with a graphical processing unit (GPU)-based SLSC approach. These results are promising for the application of deep learning to estimate correlation functions derived from ultrasound data in multiple areas of ultrasound imaging and beamforming (e.g., speckle tracking, elastography, and blood flow estimation), possibly replacing GPU-based approaches in low-power, remote, and synchronization-dependent applications.

Ultrasound is frequently used in conjunction with mammography in order to detect breast cancer as early as possible. However, due largely to the heterogeneity of breast tissue, ultrasound images are plagued with clutter that obstructs important diagnostic features. Short-lag spatial coherence (SLSC) imaging has proven to be effective at clutter reduction in noisy ultrasound images. M-Weighted SLSC and Robust-SLSC (R-SLSC) imaging were recently introduced to further improve image quality at higher lag values, while R-SLSC imaging has the added benefit of enabling the adjustment of tissue texture to produce a tissue signal-to-noise ratio (SNR) that is quantitatively similar to B-mode speckle SNR. This paper investigates the initial application of SLSC, M-Weighted SLSC, and R-SLSC imaging to nine targets in the female breast [two simple cysts, one complicated cyst, two fibroadenomas, one hematoma, one complex cystic and solid mass, one invasive ductal carcinoma (IDC), and one ductal carcinoma in situ (DCIS)]. As expected, R-SLSC beamforming improves cyst and hematoma contrast by up to 6.35 and 1.55 dB, respectively, when compared to the original B-mode image, and similar improvements are achieved with SLSC and M-Weighted SLSC imaging. However, an interesting finding from this initial investigation is that the solid masses (i.e., fibroadenoma, complex cystic and solid mass, IDC, and DCIS), which appear as hypoechoic in the B-mode image, have similarly high coherence to that of surrounding tissue in coherence-based images. This work holds promise for using SLSC, M-Weighted SLSC, and/or R-SLSC imaging to distinguish between fluid-filled and solid hypoechoic breast masses.

Several techniques and tools exist to prevent excessive bleeding during pancreatic and liver surgeries, such as lowering central venous pressure^1,2 and various clamping methods.^2,3 In addition, devices such as the harmonic scalpel and the cavitron ultrasonic surgical aspirator^3,4 control bleeding by emitting heat through vibrating at an ultrasonic frequency to coagulate the tissue and oscillating to cause explosion of cells with high water content, respectively. Despite these modern surgical advances, there is still a high risk of gastrointestinal and intra-abdominal hemorrhage and possibly death.^3,5,6 Postoperative morbidity occurs in 23% to 46% of patients who hemorrhage during liver resections, and death occurs in 4% to 5% of these patients.³ Similarly, hemorrhaging occurs in 1% to 8% of all pancreatic resections and 2% to 18% of pancreaticoduodenectomies, in addition to causing 11% to 38% of postoperative morbidity for pancreatic resections and pancreaticoduodenectomies.^5,6 Thus, there is a clinical need to distinguish major blood vessels during surgery to reduce the risk of arterial and venous injuries.