Radiographic imaging is routinely used to evaluate treatment response in solid tumors. Current imaging response metrics do not reliably predict the underlying biological response. Here, we present a multi-task deep learning approach that allows simultaneous tumor segmentation and response prediction. We design two Siamese subnetworks that are joined at multiple layers, which enables integration of multi-scale feature representations and in-depth comparison of pre-treatment and post-treatment images. The network is trained using 2568 magnetic resonance imaging scans of 321 rectal cancer patients for predicting pathologic complete response after neoadjuvant chemoradiotherapy. In multi-institution validation, the imaging-based model achieves AUC of 0.95 (95% confidence interval: 0.91–0.98) and 0.92 (0.87–0.96) in two independent cohorts of 160 and 141 patients, respectively. When combined with blood-based tumor markers, the integrated model further improves prediction accuracy with AUC 0.97 (0.93–0.99). Our approach to capturing dynamic information in longitudinal images may be broadly used for screening, treatment response evaluation, disease monitoring, and surveillance.

Background: Lymph node metastasis (LNM) in gastric cancer is a prognostic factor and has implications for the extent of lymph node dissection. The lymphatic drainage of the stomach involves multiple nodal stations with different risks of metastases. The aim of this study was to develop a deep learning system for predicting LNMs in multiple nodal stations based on preoperative CT images in patients with gastric cancer. Methods: Preoperative CT images from patients who underwent gastrectomy with lymph node dissection at two medical centres were analysed retrospectively. Using a discovery patient cohort, a system of deep convolutional neural networks was developed to predict pathologically confirmed LNMs at 11 regional nodal stations. To gain understanding about the networks' prediction ability, gradient-weighted class activation mapping for visualization was assessed. The performance was tested in an external cohort of patients by analysis of area under the receiver operating characteristic (ROC) curves (AUC), sensitivity and specificity. Results: The discovery and external cohorts included 1172 and 527 patients respectively. The deep learning system demonstrated excellent prediction accuracy in the external validation cohort, with a median AUC of 0·876 (range 0·856–0·893), sensitivity of 0·743 (0·551–0·859) and specificity of 0·936 (0·672–0·966) for 11 nodal stations. The imaging models substantially outperformed clinicopathological variables for predicting LNMs (median AUC 0·652, range 0·571–0·763). By visualizing nearly 19 000 subnetworks, imaging features related to intratumoral heterogeneity and the invasive front were found to be most useful for predicting LNMs. Conclusion: A deep learning system for the prediction of LNMs was developed based on preoperative CT images of gastric cancer. The models require further validation but may be used to inform prognosis and guide individualized surgical treatment.

Molecular phenotypic variations in metabolites offer the promise of rapid profiling of physiological and pathological states for diagnosis, monitoring, and prognosis. Since present methods are expensive, time-consuming, and still not sensitive enough, there is an urgent need for approaches that can interrogate complex biological fluids at a system-wide level. Here, we introduce hyperspectral surface-enhanced Raman spectroscopy (SERS) to profile microliters of biofluidic metabolite extraction in 15 min with a spectral set, SERSome, that can be used to describe the structures and functions of various molecules produced in the biofluid at a specific time via SERS characteristics. The metabolite differences of various biofluids, including cell culture medium and human serum, are successfully profiled, showing a diagnosis accuracy of 80.8% on the internal test set and 73% on the external validation set for prostate cancer, discovering potential biomarkers, and predicting the tissue-level pathological aggressiveness. SERSomes offer a promising methodology for metabolic phenotyping.

The complementarity and clinical utility of combining liquid biopsies and radiomic image analysis has not been demonstrated. ctDNA minimal residual disease after chemoradiotherapy (CRT) for non–small cell lung cancer (NSCLC) is highly prognostic, but on-treatment biomarkers are needed to enable response-adapted therapies. In this study, we analyzed 418 patients with NSCLC undergoing CRT to develop and validate a novel dynamic risk model that accurately predicts ultimate progression-free survival during treatment. We optimize tissue-free variant calling from plasma samples to facilitate ctDNA monitoring and demonstrate the importance of accounting for persistent clonal hematopoiesis variants. We show that mid-CRT ctDNA concentration is prognostic for disease progression and integrate additional pre-CRT risk factors, including radiomics, into a combined model that improves outcome prediction. Our results suggest that tumor features, radiomics, and mid-CRT ctDNA analysis are complementary and can identify patients at high and low risk of progression to potentially enable response-adapted therapies. Significance: This study demonstrates that combining tumor features, radiomics, and ctDNA analysis improves outcome prediction in NSCLC treated with CRT therapy. Our integrated model could enable personalized and response-adapted therapies to reduce toxicity and improve outcomes in patients.