Accurate prognosis prediction is essential for guiding cancer treatment and improving patient outcomes. While recent studies have demonstrated the potential of histopathological images in survival analysis, existing models are typically developed in a cancer-specific manner, lack extensive external validation, and often rely on molecular data that are not routinely available in clinical practice. To address these limitations, we present PROGPATH, a unified model capable of integrating histopathological image features with routinely collected clinical variables to achieve pancancer prognosis prediction. PROGPATH employs a weakly supervised deep learning architecture built upon the foundation model for image encoding. Morphological features are aggregated through an attention-guided multiple instance learning module and fused with clinical information via a cross-attention transformer. A router-based classification strategy further refines the prediction performance. PROGPATH was trained on 7999 whole-slide images (WSIs) from 6,670 patients across 15 cancer types, and extensively validated on 17 external cohorts with a total of 7374 WSIs from 4441 patients, covering 12 cancer types from 8 consortia and institutions across three continents. PROGPATH achieved consistently superior performance compared with state-of-the-art multimodal prognosis prediction models. It demonstrated strong generalizability across cancer types and robustness in stratified subgroups, including early- and advanced-stage patients, treatment cohorts (radiotherapy and pharmaceutical therapy), and biomarker-defined subsets. We further provide model interpretability by identifying pathological patterns critical to PROGPATH’s risk predictions, such as the degree of cell differentiation and extent of necrosis. Together, these results highlight the potential of PROGPATH to support pancancer outcome prediction and inform personalized cancer management strategies.
26 June 2024 in IEEE Transactions on Medical Imaging
Cervical cytology is a critical screening strategy for early detection of pre-cancerous and cancerous cervical lesions. The challenge lies in accurately classifying various cervical cytology cell types. Existing automated cervical cytology methods are primarily trained on databases covering a narrow range of coarse-grained cell types, which fail to provide a comprehensive and detailed performance analysis that accurately represents real-world cytopathology conditions. To overcome these limitations, we introduce HiCervix, the most extensive, multi-center cervical cytology dataset currently available to the public. HiCervix includes 40,229 cervical cells from 4,496 whole slide images, categorized into 29 annotated classes. These classes are organized within a three-level hierarchical tree to capture fine-grained subtype information. To exploit the semantic correlation inherent in this hierarchical tree, we propose HierSwin, a hierarchical vision transformer-based classification network. HierSwin serves as a benchmark for detailed feature learning in both coarse-level and fine-level cervical cancer classification tasks. In our comprehensive experiments, HierSwin demonstrated remarkable performance, achieving 92.08% accuracy for coarse-level classification and 82.93% accuracy averaged across all three levels. When compared to board-certified cytopathologists, HierSwin achieved high classification performance (0.8293 versus 0.7359 averaged accuracy), highlighting its potential for clinical applications. This newly released HiCervix dataset, along with our benchmark HierSwin method, is poised to make a substantial impact on the advancement of deep learning algorithms for rapid cervical cancer screening and greatly improve cancer prevention and patient outcomes in real-world clinical settings.
Super-resolution (SR) for satellite remote sensing images has been recognized as crucial and has found widespread applications across various scenarios. Previous SR methods were usually built upon Convolutional Neural Networks and Transformers, which suffer from either limited receptive fields or a lack of prior assumptions. To address these issues, we propose ESatSR, a novel SR method based on state space models. We utilize the 2D Selective Scan to obtain an enhanced capability in modeling long-range dependencies, which contributes to a wide receptive field. A Spatial Context Interaction Module (SCIM) and an Enhanced Image Reconstruction Module (EIRM) are introduced to combine image-related prior knowledge into our model, therefore guiding the process of feature extraction and reconstruction. Tailored for remote sensing images, the interaction of multi-scale spatial context and image features is leveraged to enhance the network’s capability in capturing features of small targets. Comprehensive experiments show that ESatSR demonstrates state-of-the-art performance on both OLI2MSI and RSSCN7 datasets, with the highest PSNRs of 42.11 dB and 31.42 dB, respectively. Extensive ablation studies illustrate the effectiveness of our module design.
9 December 2023 in Biomedical Signal Processing and Control
Neuroblastoma treatment planning in children relies on accurate risk stratification, but access to specialized pathology assessment and molecular testing remains limited in low- and middle-income countries. We present ANCHOR, a neuroblastoma-adapted pathology foundation model trained by self-supervised adaptation on more than 15 million H&E image patches and evaluated in 1,551 patients from six centers across two continents. ANCHOR outperformed general-purpose pathology foundation models across clinically relevant diagnostic and biomarker tasks, including histological subtype classification, Shimada classification, mitosiskaryorrhexis index grading, and prediction of risk-associated molecular alterations such as MYCN amplification, 1p36 deletion, and 11q23 deletion. Further, ANCHOR improved INRG pretreatment risk assessment and prognostication by integrating H&E-derived features with diagnostic predictions, molecular biomarker predictions, and patient age, achieving up to a 12% improvement over H&E-only features. In a randomized crossover reader study involving 15 pathologists from six centers, AI assistance increased balanced accuracy by 5%, reduced mean diagnostic time by 42%, and improved inter-rater agreement. Together, these results position ANCHOR as a scalable pathology-support framework for neuroblastoma, with potential to improve diagnostic consistency, inform molecular testing priorities, and strengthen risk-informed treatment planning from routine H&E slides.
This paper presents an improved video text detection method. At first, Harris corners detection is fulfilled to the original region. After confirming the former three text frames, this algorithm gets a suitable candidate text region according to the characteristics of these text frames and only processes candidate text region in later corner detection in order to improve detection efficiency. Then, the obtained corners are calculated by row and a local threshold is gotten based on height and density of continuous corners rows to eliminate non-text corners rows. Finally, the center location feature is employed to make sure text region further and confirm the text frames. The experiment indicates that this method has relative high detection accuracy.