Substantial progress has been made in using deep learning for cancer detection and diagnosis in medical images. Yet, there is limited success on prediction of treatment response and outcomes, which has important implications for personalized treatment strategies. A significant hurdle for clinical translation of current data-driven deep learning models is lack of interpretability, often attributable to a disconnect from the underlying pathobiology. Here, we present a biology-guided deep learning approach that enables simultaneous prediction of the tumor immune and stromal microenvironment status as well as treatment outcomes from medical images. We validate the model for predicting prognosis of gastric cancer and the benefit from adjuvant chemotherapy in a multi-center international study. Further, the model predicts response to immune checkpoint inhibitors and complements clinically approved biomarkers. Importantly, our model identifies a subset of mismatch repair-deficient tumors that are non-responsive to immunotherapy and may inform the selection of patients for combination treatments.

Defining the loss function is an important part of neural network design and critically determines the success of deep learning modeling. A significant shortcoming of the conventional loss functions is that they weight all regions in the input image volume equally, despite the fact that the system is known to be heterogeneous (i.e., some regions can achieve high prediction performance more easily than others). Here, we introduce a region-specific loss to lift the implicit assumption of homogeneous weighting for better learning. We divide the entire volume into multiple sub-regions, each with an individualized loss constructed for optimal local performance. Effectively, this scheme imposes higher weightings on the sub-regions that are more difficult to segment, and vice versa. Furthermore, the regional false positive and false negative errors are computed for each input image during a training step and the regional penalty is adjusted accordingly to enhance the overall accuracy of the prediction. Using different public and in-house medical image datasets, we demonstrate that the proposed regionally adaptive loss paradigm outperforms conventional methods in the multi-organ segmentations, without any modification to the neural network architecture or additional data preparation.

Sphingosine-1-phosphate lyase insufficiency syndrome (SPLIS) is a rare metabolic disorder caused by inactivating mutations in sphingosine-1-phosphate lyase 1 ( SGPL1 ), which is required for the final step of sphingolipid metabolism. SPLIS features include steroid-resistant nephrotic syndrome and impairment of neurological, endocrine, and hematopoietic systems. Many affected individuals die within the first 2 years. No targeted therapy for SPLIS is available. We hypothesized that SGPL1 gene replacement would address the root cause of SPLIS, thereby serving as a universal treatment for the condition. As proof of concept, we evaluated the efficacy of adeno-associated virus 9–mediated transfer of human SGPL1 (AAV-SPL) given to newborn Sgpl1 -KO mice that model SPLIS and die in the first weeks of life. Treatment dramatically prolonged survival and prevented nephrosis, neurodevelopmental delay, anemia, and hypercholesterolemia. STAT3 pathway activation and elevated proinflammatory and profibrogenic cytokines observed in KO kidneys were attenuated by treatment. Plasma and tissue sphingolipids were reduced in treated compared with untreated KO pups. SGPL1 expression and activity were measurable for at least 40 weeks. In summary, early AAV-SPL treatment prevents nephrosis, lipidosis, and neurological impairment in a mouse model of SPLIS. Our results suggest that SGPL1 gene replacement holds promise as a durable and universal targeted treatment for SPLIS.

Detection and segmentation of brain metastases (BMs) play a pivotal role in diagnosis, treatment planning, and follow-up evaluations for effective BM management. Given the rising prevalence of BM cases and its predominantly multiple onsets, automated segmentation is becoming necessary in stereotactic radiosurgery. It not only alleviates the clinician’s manual workload and improves clinical workflow efficiency but also ensures treatment safety, ultimately improving patient care. Recent strides in machine learning, particularly in deep learning (DL), have revolutionized medical image segmentation, achieving state-of-the-art results. This review aims to analyze auto-segmentation strategies, characterize the utilized data, and assess the performance of cutting-edge BM segmentation methodologies. Additionally, we delve into the challenges confronting BM segmentation and share insights gleaned from our algorithmic and clinical implementation experiences.

Brain tumors necessitate swift detection and classification for optimal patient outcomes. Deep learning has been extensively utilized to recognize complex tumor patterns in magnetic resonance imaging (MRI) images, aiding in tumor diagnosis, treatment, and prognostication. However, model complexity and limited generalizability with unfamiliar data hinder appropriate clinical integration. The objective of this study is to develop a clean-energy cloud-based deep learning platform to classify brain tumors. Three datasets of a total of 2611 axial MRI images were used to train our multi-layer convolutional neural network (CNN). Our platform automatically optimized every transfer learning and data augmentation feature combination to provide the highest predictive accuracy for our classification task. Our proposed system identified and classified brain tumors successfully and efficiently with an overall precision value of 96.8% [95% CI; 93.8–97.6]. Using clean energy supercomputing resources and cloud platforms cut our workflow to 103 min, $0 in total cost, and a negligible carbon footprint (0.0014 kg eq CO₂). By leveraging automated optimized learning, we developed a cost-effective deep learning (DL) platform that accurately classified brain tumors from axial MRI images of different levels. Although studies have identified machine learning tools to overcome these obstacles, only some are cost-effective, generalizable, and usable regardless of experience.

Background and Purpose: Intravoxel-incoherent-motion (IVIM) magnetic-resonance-imaging (MRI) and positron-emission-tomography (PET) have been investigated independently but not voxel-wise to evaluate tumor microenvironment in cervical carcinoma patients. Whether regionally combined information of IVIM and PET offers additional predictive benefit over each modality independently has not been explored. Here, we investigated parametric-response-mapping (PRM) of co-registered PET and IVIM in cervical cancer patients to identify sub-volumes that may predict tumor shrinkage to concurrent-chemoradiation-therapy (CCRT). Materials and Methods: Twenty cervical cancer patients (age: 63[41-85]) were retrospectively evaluated. Diffusion-weighted-images (DWIs) were acquired on 3.0 T MRIs using a free-breathing single-shot-spin echo-planar-imaging (EPI) sequence. Pre- and on-treatment (similar to after four-weeks of CCRT) MRI and pre-treatment FDG-PET/CT were acquired. IVIM model-fitting on the DWIs was performed using a Bayesian-fitting simplified two-compartment model. Three-dimensional rigidly-registered maps of PET/CT standardized-uptake-value (SUV) and IVIM diffusion-coefficient (D) and perfusion-fraction (f) were generated. Population-means of PET-SUV, IVIM-D and IVIM-f from pre-treatment-scans were calculated and used to generate PRM via a voxel-wise joint-histogram-analysis to classify voxels as high/low metabolic-activity and with high/low (hi/lo) cellular-density. Similar PRM maps were generated for SUV and f. Results: Tumor-volume (p < 0.001) significantly decreased, while IVIM-f (p = 0.002) and IVIM-D (p = 0.03) significantly increased on-treatment. Pre-treatment tumor-volume (r = -0.45,p = 0.04) and PRM-(SUVDlo)-D-hi (r = -0.65,p = 0.002) negatively correlated with Delta GTV, while pre-treatment IVIM-D (r = 0.64,p = 0.002), PRM-SUV(lo)f(hi) (r = 0.52,p = 0.02), and PRM-(SUVDhi)-D-lo (r = 0.74,p < 0.001) positively correlated with Delta GTV. Conclusion: IVIM and PET was performed on cervical cancer patients undergoing CCRT and we observed that both IVIM-f and IVIM-D increased during treatment. Additionally, PRM was applied, and sub-volumes were identified that were related to Delta GTV.