5 March 2025 in IEEE Journal of Biomedical and Health Informatics
Applying deep learning to predict patient prognostic survival outcomes using histological whole-slide images (WSIs) and genomic data is challenging due to the morphological and transcriptomic heterogeneity present in the tumor microenvironment. Existing deep learning-enabled methods often exhibit learning biases, primarily because the genomic knowledge used to guide directional feature extraction from WSIs may be irrelevant or incomplete. This results in a suboptimal and sometimes myopic understanding of the overall pathological landscape, potentially overlooking crucial histological insights. To tackle these challenges, we propose the CounterFactual Bidirectional Co-Attention Transformer framework. By integrating a bidirectional co-attention layer, our framework fosters effective feature interactions between the genomic and histology modalities and ensures consistent identification of prognostic features from WSIs. Using counterfactual reasoning, our model utilizes causality to model unimodal and multimodal knowledge for cancer risk stratification. This approach directly addresses and reduces bias, enables the exploration of ’what-if' scenarios, and offers a deeper understanding of how different features influence survival outcomes. Our framework, validated across eight diverse cancer benchmark datasets from The Cancer Genome Atlas (TCGA), represents a major improvement over current histology-genomic model learning methods. It shows an average 2.5% improvement in c-index performance over 18 state-of-the-art models in predicting patient prognoses across eight cancer types. Our code is released at https://github.com/BusyJzy599/CFBCT-main.
29 September 2026 in Advanced Science
Integrating histopathology, genomic, and clinical phenotypes holds promise for improving prognostic patient stratification, including within treatment‐defined subgroups, in colorectal cancer (CRC). However, ensuring the availability of all modalities in routine clinical practice remains challenging. Comprehensive molecular profiling is often constrained by cost and turnaround time, leaving histopathology as the most consistently available modality across institutions. In addition, variability in staining protocols, scanning devices, patient demographics, and outcome distributions across centers undermines the generalizability of models trained under controlled conditions with complete inputs. To address these challenges, we developed the Domain‐Adaptive Incomplete Multimodal Stratification framework (DAIMS). DAIMS leverages paired histopathology, genomic, and clinical data during training to learn generalizable disease representations and uses histology‐derived surrogate latent representations when auxiliary modalities are unavailable, enabling flexible histology‐only inference. Trained on TCGA samples and validated on two independent European and East Asian cohorts comprising 842 patients, DAIMS consistently outperformed state‐of‐the‐art methods. Adapted DAIMS improved the C‐index by 5.6%–13.9% on F1CRC and 4.1%–9.0% on SURGEN compared with baseline methods. DAIMS improved within‐cohort prognostic discrimination and localized prognostically relevant morphological patterns that remained stable across institutions. Our evaluation showed that DAIMS achieved generalizable survival stratification across disease stages, molecular subgroups, and treatment‐defined patient groups.