Early prediction of response to neoadjuvant chemotherapy (NAC) in breast cancer is crucial for guiding therapy decisions. In this work, we propose a deep learning based approach for the early NAC response prediction in ultrasound (US) imaging. We used transfer learning with deep convolutional neural networks (CNNs) to develop the response prediction models. The usefulness of two transfer learning techniques was examined. First, a CNN pre-trained on the ImageNet dataset was utilized. Second, we applied double transfer learning, the CNN pre-trained on the ImageNet dataset was additionally fine-tuned with breast mass US images to differentiate malignant and benign lesions. Two prediction tasks were investigated. First, a L1 regularized logistic regression prediction model was developed based on generic neural features extracted from US images collected before the chemotherapy (a priori prediction). Second, Siamese CNNs were used to quantify differences between US images collected before the treatment and after the first and second course of NAC. The proposed methods were evaluated using US data collected from 39 tumors. The better performing deep learning models achieved areas under the receiver operating characteristic curve of 0.797 and 0.847 in the case of the a priori prediction and the Siamese model, respectively. The proposed approach was compared with a method based on handcrafted morphological features. Our study presents the feasibility of using transfer learning with CNNs for the NAC response prediction in US imaging.

The presented studies evaluate for the first time the efficiency of tumour classification based on the quantitative analysis of ultrasound data originating from the tissue surrounding the tumour. 116 patients took part in the study after qualifying for biopsy due to suspicious breast changes. The RF signals collected from the tumour and tumour-surroundings were processed to determine quantitative measures consisting of Nakagami distribution shape parameter, entropy, and texture parameters. The utility of parameters for the classification of benign and malignant lesions was assessed in relation to the results of histopathology. The best multi-parametric classifier reached an AUC of 0.92 and of 0.83 for outer and intra-tumour data, respectively. A classifier composed of two types of parameters, parameters based on signals scattered in the tumour and in the surrounding tissue, allowed the classification of breast changes with sensitivity of 93%, specificity of 88%, and AUC of 0.94. Among the 4095 multi-parameter classifiers tested, only in eight cases the result of classification based on data from the surrounding tumour tissue was worse than when using tumour data. The presented results indicate the high usefulness of QUS analysis of echoes from the tissue surrounding the tumour in the classification of breast lesions.

Neoadjuvant chemotherapy (NAC) is used in patients with breast cancer to reduce tumor focus, metastatic risk, and patient mortality. Monitoring NAC effects is necessary to capture resistant patients and stop or change treatment. The existing methods for evaluating NAC results have some limitations. The aim of this study was to assess the tumor response at an early stage, after the first doses of the NAC, based on the variability of the backscattered ultrasound energy, and backscatter statistics. The backscatter statistics has not previously been used to monitor NAC effects. The B-mode ultrasound images and raw radio frequency data from breast tumors were obtained using an ultrasound scanner before chemotherapy and 1 week after each NAC cycle. The study included twenty-four malignant breast cancers diagnosed in sixteen patients and qualified for neoadjuvant treatment before surgery. The shape parameter of the homodyned K distribution and integrated backscatter, along with the tumor size in the longest dimension, were determined based on ultrasound data and used as markers for NAC response. Cancer tumors were assigned to responding and non-responding groups, according to histopathological evaluation, which was a reference in assessing the utility of markers. Statistical analysis was performed to rate the ability of markers to predict the final NAC response based on data obtained after subsequent therapeutic doses. Statistically significant differences (p<0.05) between groups were obtained after 2, 3, 4, and 5 doses of NAC for quantitative ultrasound markers and after 5 doses for the assessment based on maximum tumor dimension. Statistical analysis showed that, after the second and third NAC courses the classification based on integrated backscatter marker was characterized by an AUC of 0.69 and 0.82, respectively. The introduction of the second quantitative marker describing the statistical properties of scattering increased the corresponding AUC values to 0.82 and 0.91. Quantitative ultrasound information can characterize the tumor's pathological response better and at an earlier stage of therapy than the assessment of the reduction of its dimensions. The introduction of statistical parameters of ultrasonic backscatter to monitor the effects of chemotherapy can increase the effectiveness of monitoring and contribute to a better personalization of NAC therapy.

Objective . Prediction of the response to neoadjuvant chemotherapy (NAC) in breast cancer is important for patient outcomes. In this work, we propose a deep learning based approach to NAC response prediction in ultrasound (US) imaging. Approach. We develop recurrent neural networks that can process serial US imaging data to predict chemotherapy outcomes. We present models that can process either raw radio-frequency (RF) US data or regular US images. The proposed approach is evaluated based on 204 sequences of US data from 51 breast cancers. Each sequence included US data collected before the chemotherapy and after each subsequent dose, up to the 4th course. We investigate three pre-trained convolutional neural networks (CNNs) as back-bone feature extractors for the recurrent network. The CNNs were pre-trained using raw US RF data, US b-mode images and RGB images from the ImageNet dataset. The first two networks were developed using US data collected from malignant and benign breast masses. Main results . For the pre-treatment data, the better performing network, with back-bone CNN pre-trained on US images, achieved area under the receiver operating curve (AUC) of 0.81 (±0.04). Performance of the recurrent networks improved with each course of the chemotherapy. For the 4th course, the better performing model, based on the CNN pre-trained with RGB images, achieved AUC value of 0.93 (±0.03). Statistical analysis based on the DeLong test presented that there were no significant differences in AUC values between the pre-trained networks at each stage of the chemotherapy ( p -values > 0.05). Significance . Our study demonstrates the feasibility of using recurrent neural networks for the NAC response prediction in breast cancer US.