10 February 2020 in IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control
In this paper, two novel deep learning methods are proposed for displacement estimation in ultrasound elastography. Although Convolutional Neural Networks (CNN) have been very successful for displacement estimation in computer vision, they have been rarely used for ultrasound elastography. One of the main limitations is that the Radio Frequency (RF) ultrasound data, which is crucial for precise displacement estimation, has vastly different frequency characteristics compared to images in computer vision. Top-rank CNN methods used in computer vision applications are mostly based on a multi-level strategy which estimates finer resolution based on coarser ones. This strategy does not work well for RF data due to its large high frequency content. To mitigate the problem, we propose Modified Pyramid, Warping and Cost volume Network (MPWC-Net) and RFMPWC-Net, both based on PWC-Net, to exploit information in RF data by employing two different strategies. We obtained promising results using networks trained only on computer vision images. In the next step, we constructed a large ultrasound simulation database, and proposed a new loss function to fine-tune the network to improve its performance. The proposed networks and well-known optical flow networks as well as state-of-the-art elastography methods are evaluated using simulation, phantom and in vivo data. Our two proposed networks substantially outperform current deep learning methods in terms of Contrast to Noise Ratio (CNR) and Strain Ratio (SR). Also, the proposed methods perform similar to the state-of-the-art elastography methods in terms of CNR and have better SR by substantially reducing the underestimation bias.
27 January 2022 in IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control
The performance of ultrasound elastography (USE) heavily depends on the accuracy of displacement estimation. Recently, convolutional neural networks (CNNs) have shown promising performance in optical flow estimation and have been adopted for USE displacement estimation. Networks trained on computer vision images are not optimized for USE displacement estimation since there is a large gap between the computer vision images and the high-frequency radio frequency (RF) ultrasound data. Many researchers tried to adopt the optical flow CNNs to USE by applying transfer learning to improve the performance of CNNs for USE. However, the ground-truth displacement in real ultrasound data is unknown, and simulated data exhibit a domain shift compared to the real data and are also computationally expensive to generate. To resolve this issue, semisupervised methods have been proposed in which the networks pretrained on computer vision images are fine-tuned using real ultrasound data. In this article, we employ a semisupervised method by exploiting the first- and second-order derivatives of the displacement field for regularization. We also modify the network structure to estimate both forward and backward displacements and propose to use consistency between the forward and backward strains as an additional regularizer to further enhance the performance. We validate our method using several experimental phantom and in vivo data. We also show that the network fine-tuned by our proposed method using experimental phantom data performs well on in vivo data similar to the network fine-tuned on in vivo data. Our results also show that the proposed method outperforms current deep learning methods and is comparable to computationally expensive optimization-based algorithms.
19 December 2022 in IEEE Transactions on Medical Imaging
Convolutional Neural Networks (CNN) have shown promising results for displacement estimation in UltraSound Elastography (USE). Many modifications have been proposed to improve the displacement estimation of CNNs for USE in the axial direction. However, the lateral strain, which is essential in several downstream tasks such as the inverse problem of elasticity imaging, remains a challenge. The lateral strain estimation is complicated since the motion and the sampling frequency in this direction are substantially lower than the axial one, and a lack of carrier signal in this direction. In computer vision applications, the axial and the lateral motions are independent. In contrast, the tissue motion pattern in USE is governed by laws of physics which link the axial and lateral displacements. In this paper, inspired by Hooke’s law, we first propose Physically Inspired ConsTraint for Unsupervised Regularized Elastography (PICTURE), where we impose a constraint on the Effective Poisson’s ratio (EPR) to improve the lateral strain estimation. In the next step, we propose self-supervised PICTURE (sPICTURE) to further enhance the strain image estimation. Extensive experiments on simulation, experimental phantom and in vivo data demonstrate that the proposed methods estimate accurate axial and lateral strain maps.
26 April 2021 in IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control
Quantitative ultrasound (QUS) can reveal crucial information on tissue properties, such as scatterer density. If the scatterer density per resolution cell is above or below 10, the tissue is considered as fully developed speckle (FDS) or underdeveloped speckle (UDS), respectively. Conventionally, the scatterer density has been classified using estimated statistical parameters of the amplitude of backscattered echoes. However, if the patch size is small, the estimation is not accurate. These parameters are also highly dependent on imaging settings. In this article, we adapt convolutional neural network (CNN) architectures for QUS and train them using simulation data. We further improve the network's performance by utilizing patch statistics as additional input channels. Inspired by deep supervision and multitask learning, we propose a second method to exploit patch statistics. We evaluate the networks using simulation data and experimental phantoms. We also compare our proposed methods with different classic and deep learning models and demonstrate their superior performance in the classification of tissues with different scatterer density values. The results also show that we are able to classify scatterer density in different imaging parameters with no need for a reference phantom. This work demonstrates the potential of CNNs in classifying scatterer density in ultrasound images.
2 June 2023 in IEEE Transactions on Medical Imaging
Tracking the displacement between the pre- and post-deformed radio-frequency (RF) frames is a pivotal step of ultrasound elastography, which depicts tissue mechanical properties to identify pathologies. Due to ultrasound’s poor ability to capture information pertaining to the lateral direction, the existing displacement estimation techniques fail to generate an accurate lateral displacement or strain map. The attempts made in the literature to mitigate this well-known issue suffer from one of the following limitations: 1) Sampling size is substantially increased, rendering the method computationally and memory expensive. 2) The lateral displacement estimation entirely depends on the axial one, ignoring data fidelity and creating large errors. This paper proposes exploiting the effective Poisson’s ratio (EPR)-based mechanical correspondence between the axial and lateral strains along with the RF data fidelity and displacement continuity to improve the lateral displacement and strain estimation accuracies. We call our techniques MechSOUL (Mechanically-constrained Second-Order Ultrasound eLastography) and L 1-MechSOUL ( L 1-norm-based MechSOUL), which optimize L 2- and L 1-norm-based penalty functions, respectively. Extensive validation experiments with simulated, phantom, and in vivo datasets demonstrate that MechSOUL and L 1-MechSOUL’s lateral strain and EPR estimation abilities are substantially superior to those of the recently-published elastography techniques. We will publish the MATLAB codes of MechSOUL and L 1--MechSOUL at http://code.sonography.ai after the acceptance of this paper.