A deep learning technique to enhance 3D images of the complex-valued permittivity of the breast obtained via microwave imaging is investigated. The developed technique is an extension of one created to enhance 2D images. We employ a 3D Convolutional Neural Network, based on the U-Net architecture, that takes in 3D images obtained using the Contrast-Source Inversion (CSI) method and attempts to produce the true 3D image of the permittivity. The training set consists of 3D CSI images, along with the true numerical phantom images from which the microwave scattered field utilized to create the CSI reconstructions was synthetically generated. Each numerical phantom varies with respect to the size, number, and location of tumors within the fibroglandular region. The reconstructed permittivity images produced by the proposed 3D U-Net show that the network is not only able to remove the artifacts that are typical of CSI reconstructions, but it also enhances the detectability of the tumors. We test the trained U-Net with 3D images obtained from experimentally collected microwave data as well as with images obtained synthetically. Significantly, the results illustrate that although the network was trained using only images obtained from synthetic data, it performed well with images obtained from both synthetic and experimental data. Quantitative evaluations are reported using Receiver Operating Characteristics (ROC) curves for the tumor detectability and RMS error for the enhancement of the reconstructions.
We present a deep learning method used in conjunction with dual-modal microwave-ultrasound imaging to produce tomographic reconstructions of the complex-valued permittivity of numerical breast phantoms. We also assess tumor segmentation performance using the reconstructed permittivity as a feature. The contrast source inversion (CSI) technique is used to create the complex-permittivity images of the breast with ultrasound-derived tissue regions utilized as prior information. However, imaging artifacts make the detection of tumors difficult. To overcome this issue we train a convolutional neural network (CNN) that takes in, as input, the dual-modal CSI reconstruction and attempts to produce the true image of the complex tissue permittivity. The neural network consists of successive convolutional and downsampling layers, followed by successive deconvolutional and upsampling layers based on the U-Net architecture. To train the neural network, the input-output pairs consist of CSI’s dual-modal reconstructions, along with the true numerical phantom images from which the microwave scattered field was synthetically generated. The reconstructed permittivity images produced by the CNN show that the network is not only able to remove the artifacts that are typical of CSI reconstructions, but can also improve the detectability of tumors. The performance of the CNN is assessed using a four-fold cross-validation on our dataset that shows improvement over CSI both in terms of reconstruction error and tumor segmentation performance.
30 April 2021 in IEEE Journal on Multiscale and Multiphysics Computational Techniques
Combined ultrasound-microwave breast imaging requires a mechanism to guide one imaging modality using the other. To this end, a convolutional neural network (CNN) is proposed for the mapping of ultrasound property images to dielectric property images for combined ultrasound-microwave breast imaging applications. In this approach, higher resolution ultrasound images are obtained by inverting the ultrasound scattered pressure data using a linearized inverse scattering algorithm. The reconstructed Born-based ultrasound compressibility images are then used to predict dielectric images for the same breast phantoms. These predicted dielectric images can then be used to guide microwave imaging reconstruction to achieve higher accuracy images. To this end, a CNN is trained based on the input of the reconstructed quantitative ultrasonic compressibility and the output of the true quantitative dielectric properties corresponding to the same numerical phantom. Several numerical MRI-derived breast phantoms are used to train and test this CNN. The predicted dielectric properties are tested using different numerical MRI-derived breast phantoms and the predicted profiles show promising results. The predicted dielectric properties are also used as the initial guess prior for the microwave inversion algorithm which leads to the enhancement of the reconstruction of dielectric properties as compared to the blind microwave inversion.
A two-stage workflow for detecting and monitoring tumors in the human breast with an inverse scattering-based technique is presented. Stage 1 involves a phaseless bulk-parameter inference neural network that recovers the geometry and permittivity of the breast fibroglandular region. The bulk parameters are used for calibration and as prior information for Stage 2, a full phase contrast source inversion of the measurement data, to detect regions of high relative complex-valued permittivity in the breast based on an assumed known overall tissue geometry. We demonstrate the ability of the workflow to recover the geometry and bulk permittivity of the different sized fibroglandular regions, and to detect and localize tumors of various sizes and locations within the breast model. Preliminary results show promise for a synthetically trained Stage 1 network to be applied to experimental data and provide quality prior information in practical imaging situations.
16 April 2021 in Neural Computing and Applications
A novel deep convolutional fusion architecture with multiple branches is proposed to address the challenges posed by electromagnetic inverse scattering problems. Traditional inverse scattering techniques suffer from various difficulties, including strong ill-conditioning, high computational cost, and intrinsic nonlinearity. To overcome these challenges, a multi-branch convolutional neural network (CNN) is proposed for reconstructing two-dimensional (2D) images of breast permittivity in order to detect tumors. The proposed architecture, which is influenced by objective-function methods, inputs scattered-field data and ultrasound-derived breast mask as prior information to reconstruct images of permittivity. The architecture consists of two branches, a decoder-only branch and a convolutional branch, to handle inputs of different formats. The final reconstructed 2D image is obtained by fusing the outputs of these two branches. The CNN is trained using eight numerical MRI-based breast models. Results demonstrate that the proposed approach provides high-quality imaging for high-contrast objects of interest. This work opens up a new avenue for real-time quantitative microwave imaging using deep learning.