A deep learning technique to enhance 3D images of the complex-valued permittivity of the breast obtained via microwave imaging is investigated. The developed technique is an extension of one created to enhance 2D images. We employ a 3D Convolutional Neural Network, based on the U-Net architecture, that takes in 3D images obtained using the Contrast-Source Inversion (CSI) method and attempts to produce the true 3D image of the permittivity. The training set consists of 3D CSI images, along with the true numerical phantom images from which the microwave scattered field utilized to create the CSI reconstructions was synthetically generated. Each numerical phantom varies with respect to the size, number, and location of tumors within the fibroglandular region. The reconstructed permittivity images produced by the proposed 3D U-Net show that the network is not only able to remove the artifacts that are typical of CSI reconstructions, but it also enhances the detectability of the tumors. We test the trained U-Net with 3D images obtained from experimentally collected microwave data as well as with images obtained synthetically. Significantly, the results illustrate that although the network was trained using only images obtained from synthetic data, it performed well with images obtained from both synthetic and experimental data. Quantitative evaluations are reported using Receiver Operating Characteristics (ROC) curves for the tumor detectability and RMS error for the enhancement of the reconstructions.

The resolution of an experimental microwave tomography (MWT) system is investigated. Using two cylindrical nylon targets and an operating frequency of 5 GHz, a separation resolution of 2 mm, or 1/30 of a wavelength, is achieved. While this resolution is among the highest reported in the literature, it is not a sufficiently robust indicator of the expected resolution obtainable for complex targets, and this is shown with further examples of more complicated targets. However, the basic separation resolution limit obtained is a good way of comparing various aspects of different MWT systems.

We present a deep learning method used in conjunction with dual-modal microwave-ultrasound imaging to produce tomographic reconstructions of the complex-valued permittivity of numerical breast phantoms. We also assess tumor segmentation performance using the reconstructed permittivity as a feature. The contrast source inversion (CSI) technique is used to create the complex-permittivity images of the breast with ultrasound-derived tissue regions utilized as prior information. However, imaging artifacts make the detection of tumors difficult. To overcome this issue we train a convolutional neural network (CNN) that takes in, as input, the dual-modal CSI reconstruction and attempts to produce the true image of the complex tissue permittivity. The neural network consists of successive convolutional and downsampling layers, followed by successive deconvolutional and upsampling layers based on the U-Net architecture. To train the neural network, the input-output pairs consist of CSI’s dual-modal reconstructions, along with the true numerical phantom images from which the microwave scattered field was synthetically generated. The reconstructed permittivity images produced by the CNN show that the network is not only able to remove the artifacts that are typical of CSI reconstructions, but can also improve the detectability of tumors. The performance of the CNN is assessed using a four-fold cross-validation on our dataset that shows improvement over CSI both in terms of reconstruction error and tumor segmentation performance.

The tumor detection performance of two air-based metallic microwave breast imaging systems is evaluated experimentally using breast phantoms having realistic tissue permittivities. The first system has a circular-cylindrical shape, whereas the second has a faceted hemispherical shape. No immersion medium is utilized making these systems low-loss quasi-resonant structures. The breast phantoms are composed of three distinct volumes representing structurally simplified adipose, fibroglandular, and tumor regions filled with tissue-mimicking fluids. Scattered-field data are collected, at multiple frequencies, on the inside surface of the chambers: the normal component of the electric field at 18 locations in the cylindrical chamber and 24 tangential magnetic field components in the faceted chamber. Quantitative 3-D imaging of these high-contrast phantoms is performed utilizing the finite-element contrast-source-inversion algorithm wherein an inhomogeneous numerical background, constituting prior information, is introduced. Experiments are conducted with various accuracies of this prior information, both in terms of the permittivity that is used as well as the accuracy of the geometry compared with the true regions within the phantom. Image segmentation techniques based on thresholding the 3-D reconstructed images are evaluated for tumor detection. Utilizing receiver operating characteristic (ROC) curves, it is shown that taking the intersection of multifrequency thresholded 3-D images performs the best for detecting tumors.

A deep learning approach is proposed for performing tissue-type classification of tomographic microwave and ultrasound property images of the breast. The approach is based on a convolutional neural network (CNN) utilizing the U-net architecture that also quantifies the uncertainty in the classification of each pixel. Quantitative tomographic reconstructions of dielectric properties (complex-valued permittivity), ultrasonic properties (compressibility and attenuation), as well as their combination, with the corresponding actual tissue-type classification constitute the training set. The CNN learns to map the quantitative property reconstructions to a single tissue-type image. The level of confidence in predicting a tissue-type at each pixel is determined. This uncertainty quantification is diagnostically critical for biomedical applications, especially when attempting to distinguish between cancerous and healthy tissues. The Gauss-Newton Inversion algorithm is used for the quantitative reconstruction of both dielectric and ultrasonic properties. Electromagnetic and ultrasound scattered-field data is obtained from MRI-derived numerical breast phantoms. Several numerical breast phantoms types, from fatty to dense, are considered. The proposed classification and uncertainty quantification approach is shown to outperform a previously studied tissue-type classification method based on a Bayesian approach.