One way to resolve the problem of scarce and expensive data in deep learning for medical applications is using transfer learning and fine-tuning a network which has been trained on a large dataset. The common practice in transfer learning is to keep the shallow layers unchanged and to modify deeper layers according to the new dataset. This approach may not work when using a U-Net and when moving from a different domain to ultrasound (US) images due to their drastically different appearance. In this study, we investigated the effect of fine-tuning different sets of layers of a pre-trained U-Net for US image segmentation. Two different schemes were analyzed, based on two different definitions of shallow and deep layers. We studied simulated US images, as well as two human US datasets. We also included a chest X-ray dataset. The results showed that choosing which layers to fine-tune is a critical task. In particular, they demonstrated that fine-tuning the last layers of the network, which is the common practice for classification networks, is often the worst strategy. It may therefore be more appropriate to fine-tune the shallow layers rather than deep layers in US image segmentation when using a U-Net. Shallow layers learn lower level features which are critical in automatic segmentation of medical images. Even when a large US dataset is available, we also observed that fine-tuning shallow layers is a faster approach compared to fine-tuning the whole network.

A critical step in quasi-static ultrasound elastography is the estimation of time delay between two frames of radio-frequency (RF) data that are obtained while the tissue is undergoing deformation. This paper presents a novel technique for time-delay estimation (TDE) of all samples of RF data simultaneously, thereby exploiting all the information in RF data for TDE. A nonlinear cost function that incorporates similarity of RF data intensity and prior information of displacement continuity is formulated. Optimization of this function involves searching for TDE of all samples of the RF data, rendering the optimization intractable with conventional techniques given that the number of variables can be approximately one million. Therefore, the optimization problem is converted to a sparse linear system of equations, and is solved in real time using a computationally efficient optimization technique. We call our method GLobal Ultrasound Elastography (GLUE), and compare it to dynamic programming analytic minimization (DPAM) and normalized cross correlation (NCC) techniques. Our simulation results show that the contrast-to-noise ratio (CNR) values of the axial strain maps are 4.94 for NCC, 14.62 for DPAM, and 26.31 for GLUE. Our results on experimental data from tissue mimicking phantoms show that the CNR values of the axial strain maps are 1.07 for NCC, 16.01 for DPAM, and 18.21 for GLUE. Finally, our results on in vivo data show that the CNR values of the axial strain maps are 3.56 for DPAM and 13.20 for GLUE.

Deep learning for ultrasound image formation is rapidly garnering research support and attention, quickly rising as the latest frontier in ultrasound image formation, with much promise to balance both image quality and display speed. Despite this promise, one challenge with identifying optimal solutions is the absence of unified evaluation methods and datasets that are not specific to a single research group. This article introduces the largest known international database of ultrasound channel data and describes the associated evaluation methods that were initially developed for the challenge on ultrasound beamforming with deep learning (CUBDL), which was offered as a component of the 2020 IEEE International Ultrasonics Symposium. We summarize the challenge results and present qualitative and quantitative assessments using both the initially closed CUBDL evaluation test dataset (which was crowd-sourced from multiple groups around the world) and additional in vivo breast ultrasound data contributed after the challenge was completed. As an example quantitative assessment, single plane wave images from the CUBDL Task 1 dataset produced a mean generalized contrast-to-noise ratio (gCNR) of 0.67 and a mean lateral resolution of 0.42 mm when formed with delay-and-sum beamforming, compared with a mean gCNR as high as 0.81 and a mean lateral resolution as low as 0.32 mm when formed with networks submitted by the challenge winners. We also describe contributed CUBDL data that may be used for training of future networks. The compiled database includes a total of 576 image acquisition sequences. We additionally introduce a neural-network-based global sound speed estimator implementation that was necessary to fairly evaluate the results obtained with this international database. The integration of CUBDL evaluation methods, evaluation code, network weights from the challenge winners, and all datasets described herein are publicly available (visit https://cubdl.jhu.edu for details).

In this paper, two novel deep learning methods are proposed for displacement estimation in ultrasound elastography. Although Convolutional Neural Networks (CNN) have been very successful for displacement estimation in computer vision, they have been rarely used for ultrasound elastography. One of the main limitations is that the Radio Frequency (RF) ultrasound data, which is crucial for precise displacement estimation, has vastly different frequency characteristics compared to images in computer vision. Top-rank CNN methods used in computer vision applications are mostly based on a multi-level strategy which estimates finer resolution based on coarser ones. This strategy does not work well for RF data due to its large high frequency content. To mitigate the problem, we propose Modified Pyramid, Warping and Cost volume Network (MPWC-Net) and RFMPWC-Net, both based on PWC-Net, to exploit information in RF data by employing two different strategies. We obtained promising results using networks trained only on computer vision images. In the next step, we constructed a large ultrasound simulation database, and proposed a new loss function to fine-tune the network to improve its performance. The proposed networks and well-known optical flow networks as well as state-of-the-art elastography methods are evaluated using simulation, phantom and in vivo data. Our two proposed networks substantially outperform current deep learning methods in terms of Contrast to Noise Ratio (CNR) and Strain Ratio (SR). Also, the proposed methods perform similar to the state-of-the-art elastography methods in terms of CNR and have better SR by substantially reducing the underestimation bias.

In brain tumor surgery, the quality and safety of the procedure can be impacted by intra-operative tissue deformation, called brain shift. Brain shift can move the surgical targets and other vital structures such as blood vessels, thus invalidating the pre-surgical plan. Intra-operative ultrasound (iUS) is a convenient and cost-effective imaging tool to track brain shift and tumor resection. Accurate image registration techniques that update pre-surgical MRI based on iUS are crucial but challenging. The MICCAI Challenge 2018 for Correction of Brain shift with Intra-Operative UltraSound (CuRIOUS2018) provided a public platform to benchmark MRI-iUS registration algorithms on newly released clinical datasets. In this work, we present the data, setup, evaluation, and results of CuRIOUS 2018, which received 6 fully automated algorithms from leading academic and industrial research groups. All algorithms were first trained with the public RESECT database, and then ranked based on a test dataset of 10 additional cases with identical data curation and annotation protocols as the RESECT database. The article compares the results of all participating teams and discusses the insights gained from the challenge, as well as future work.

One of the main limitations of ultrasound imaging is that image quality and interpretation depend on the skill of the user and the experience of the clinician. Quantitative ultrasound (QUS) methods provide objective, system-independent estimates of tissue properties such as acoustic attenuation and backscattering properties of tissue, which are valuable as objective tools for both diagnosis and intervention. Accurate and precise estimation of these properties requires correct compensation for intervening tissue attenuation. Prior attempts to estimate interveningtissues attenuation based on minimizing cost functions that compared backscattered echo data to models have resulted in limited precision and accuracy. To overcome these limitations, in this work we incorporate the prior information of piece-wise continuity of QUS parameters as a regularization term into our cost function. We further propose to calculate this cost function using Dynamic Programming (DP), a computationally efficient optimization algorithm that finds the global optimum. Our results on tissue-mimicking phantoms show that DP substantially outperforms a published Least Squares Method in terms of both estimation bias and variance.