Intra-tumoral heterogeneity and cell-state plasticity are key drivers for the therapeutic resistance of glioblastoma. Here, we investigate the association between spatial cellular organization and glioblastoma prognosis. Leveraging single-cell RNA-seq and spatial transcriptomics data, we develop a deep learning model to predict transcriptional subtypes of glioblastoma cells from histology images. Employing this model, we phenotypically analyze 40 million tissue spots from 410 patients and identify consistent associations between tumor architecture and prognosis across two independent cohorts. Patients with poor prognosis exhibit higher proportions of tumor cells expressing a hypoxia-induced transcriptional program. Furthermore, a clustering pattern of astrocyte-like tumor cells is associated with worse prognosis, while dispersion and connection of the astrocytes with other transcriptional subtypes correlate with decreased risk. To validate these results, we develop a separate deep learning model that utilizes histology images to predict prognosis. Applying this model to spatial transcriptomics data reveal survival-associated regional gene expression programs. Overall, our study presents a scalable approach to unravel the transcriptional heterogeneity of glioblastoma and establishes a critical connection between spatial cellular architecture and clinical outcomes.

Cancer is a heterogeneous disease requiring costly genetic profiling for better understanding and management. Recent advances in deep learning have enabled cost-effective predictions of genetic alterations from whole slide images (WSIs). While transformers have driven significant progress in non-medical domains, their application to WSIs lags behind due to high model complexity and limited dataset sizes. Here, we introduce SEQUOIA, a linearized transformer model that predicts cancer transcriptomic profiles from WSIs. SEQUOIA is developed using 7584 tumor samples across 16 cancer types, with its generalization capacity validated on two independent cohorts comprising 1368 tumors. Accurately predicted genes are associated with key cancer processes, including inflammatory response, cell cycles and metabolism. Further, we demonstrate the value of SEQUOIA in stratifying the risk of breast cancer recurrence and in resolving spatial gene expression at loco-regional levels. SEQUOIA hence deciphers clinically relevant information from WSIs, opening avenues for personalized cancer management.

Non–small cell lung cancer (NSCLC) constitutes over 80% of lung cancer cases and remains a leading cause of cancer-related mortality worldwide. Despite the advent of immune checkpoint inhibitors, their efficacy is limited to 27 to 45% of patients. Identifying likely treatment responders is essential for optimizing healthcare and improving quality of life. We generated multiplex immunofluorescence (mIF) images, histopathology, and RNA sequencing data from human NSCLC tissues. Through the analysis of mIF images, we characterized the spatial organization of 1.5 million cells based on the expression levels for 33 biomarkers. To enable large-scale characterization of tumor microenvironments, we developed NucSegAI, a deep learning model for automated nuclear segmentation and cellular classification in histology images. With this model, we analyzed the morphological, textural, and topological phenotypes of 45.6 million cells across 119 whole-slide images. Through unsupervised phenotype discovery, we identified specific lymphocyte phenotypes predictive of immunotherapy response. Our findings can improve patient stratification and guide selection of effective therapeutic regimens.

Diet is a critical environmental factor affecting breast cancer risk, and recent evidence shows that dietary exposures during early development can affect lifetime mammary cancer susceptibility. To elucidate the underlying mechanisms, we used our established crossover feeding mouse model, where exposure to a high-fat and high-sugar (HFHS) diet during defined developmental windows determines mammary tumor incidence and latency in carcinogen-treated mice. Mammary tumor incidence is significantly increased in mice receiving a HFHS post-weaning diet (high-tumor mice, HT) compared to those receiving a HFHS diet during gestation (low-tumor mice, LT). The current study revealed that the mammary stem cell (MaSC) population was significantly increased in mammary glands from HT compared to LT mice. Igf1 expression was increased in mammary stromal cells from HT mice, where it promoted MaSC self-renewal. The increased Igf1 expression was induced by DNA hypomethylation of the Igf1 Pr1 promoter, mediated by a decrease in Dnmt3b levels. Mammary tissues from HT mice also had reduced levels of Igfbp5, leading to increased bioavailability of tissue Igf1. This study provides novel insights into how early dietary exposures program mammary cancer risk, demonstrating that effective dietary intervention can reduce mammary cancer incidence.

Cancer is a heterogeneous disease that demands precise molecular profiling for better understanding and management. Recently, deep learning has demonstrated potentials for cost-efficient prediction of molecular alterations from histology images. While transformer-based deep learning architectures have enabled significant progress in non-medical domains, their application to histology images remains limited due to small dataset sizes coupled with the explosion of trainable parameters. Here, we develop SEQUOIA , a transformer model to predict cancer transcriptomes from whole-slide histology images. To enable the full potential of transformers, we first pre-train the model using data from 1,802 normal tissues. Then, we fine-tune and evaluate the model in 4,331 tumor samples across nine cancer types. The prediction performance is assessed at individual gene levels and pathway levels through Pearson correlation analysis and root mean square error. The generalization capacity is validated across two independent cohorts comprising 1,305 tumors. In predicting the expression levels of 25,749 genes, the highest performance is observed in cancers from breast, kidney and lung, where SEQUOIA accurately predicts the expression of 11,069, 10,086 and 8,759 genes, respectively. The accurately predicted genes are associated with the regulation of inflammatory response, cell cycles and metabolisms. While the model is trained at the tissue level, we showcase its potential in predicting spatial gene expression patterns using spatial transcriptomics datasets. Leveraging the prediction performance, we develop a digital gene expression signature that predicts the risk of recurrence in breast cancer. SEQUOIA deciphers clinically relevant gene expression patterns from histology images, opening avenues for improved cancer management and personalized therapies.