30 April 2024 in IEEE Transactions on Medical Imaging
Accurate segmentation of anatomical structures in Computed Tomography (CT) images is crucial for clinical diagnosis, treatment planning, and disease monitoring. The present deep learning segmentation methods are hindered by factors such as data scale and model size. Inspired by how doctors identify tissues, we propose a novel approach, the Prior Category Network (PCNet), that boosts segmentation performance by leveraging prior knowledge between different categories of anatomical structures. Our PCNet comprises three key components: prior category prompt (PCP), hierarchy category system (HCS), and hierarchy category loss (HCL). PCP utilizes Contrastive Language-Image Pretraining (CLIP), along with attention modules, to systematically define the relationships between anatomical categories as identified by clinicians. HCS guides the segmentation model in distinguishing between specific organs, anatomical structures, and functional systems through hierarchical relationships. HCL serves as a consistency constraint, fortifying the directional guidance provided by HCS to enhance the segmentation model’s accuracy and robustness. We conducted extensive experiments to validate the effectiveness of our approach, and the results indicate that PCNet can generate a high-performance, universal model for CT segmentation. The PCNet framework also demonstrates a significant transferability on multiple downstream tasks. The ablation experiments show that the methodology employed in constructing the HCS is of critical importance. The prompt and HCS can be accessed at https://github.com/YixinChen-AI/PCNet.
7 January 2025 in Health Data Science
Background: Source-free unsupervised domain adaptation (SFUDA) methods aim to address the challenge of domain shift while preserving data privacy. Existing SFUDA approaches construct reliable and confident pseudo-labels for target-domain data through denoising methods, thereby guiding the training of the target-domain model. The effectiveness of denoising approaches is influenced by the degree of domain gap between the source and target domains. A marked shift can cause the pseudo-labels to be unreliable, even after applying denoising. Methods: We propose a novel 2-stage framework for SFUDA called visual prompt source-free domain adaptation (VP-SFDA). We propose input-specific visual prompt in the first stage, prompting process, which bridges the target-domain data to source-domain distribution. Our method utilizes visual prompts and batch normalization constraint to enable the alignment model to learn domain-specific knowledge and align the target-domain data with the source-domain contribution. The second stage is the adaptation process, which aims at optimizing the segmentation model from the source domain to the target domain. This is accomplished through the denoising techniques, ultimately enhancing the performance. Results: Our study presents a comparative analysis of several SFUDA techniques in the VP-SFDA framework across 4 tasks: abdominal magnetic resonance imaging (MRI) to computed tomography (CT), abdominal CT to MRI, cardiac MRI to CT, and cardiac CT to MRI. Notably, in the abdominal MRI to CT adaptation task, the VP-OS method achieved a remarkable improvement, increasing the average DICE score from 0.658 to 0.773 ( P 0.01) and reducing the average surface distance (ASD) from 3.489 to 2.961 ( P 0.01). Similarly, the VP-LD and VP-DPL methods also showed significant improvements over their base algorithms in both abdominal and cardiac MRI to CT tasks. Conclusions: This paper proposes VP-SFDA, a novel 2-stage framework for SFUDA in medical imaging, which achieves superior performance through input-specific visual prompts and batch normalization constraint for domain adaptation, coupled with denoising methods for enhanced results. Comparative experiments on 4 medical SFUDA tasks demonstrate that VO-SFDA surpasses existing methods, with ablation studies confirming the benefits of domain-specific patterns.
24 October 2025 in Nature Communications
The integration of deep learning in medical imaging has significantly advanced diagnostic, therapeutic, and research outcomes. However, applying universal models across multiple modalities remains challenging due to inherent inter-modality variability. Here we present the Modality Projection Universal Model (MPUM), trained on 861 subjects, which dynamically adapts to diverse imaging modalities through a modality-projection strategy. MPUM achieves state-of-the-art, whole-body organ segmentation, providing rapid localization for computer-aided diagnosis and precise anatomical quantification to support clinical decision-making. A controller-based convolutional layer further enables saliency map visualization, enhancing model interpretability for clinical use. Beyond segmentation, MPUM reveals metabolic correlations along the brain-body axis and between distinct brain regions, providing insights into systemic and physiological interactions from a whole-body perspective. Here we show that this universal framework accelerates diagnosis, facilitates large-scale imaging analysis, and bridges anatomical and metabolic information, enabling discovery of cross-organ disease mechanisms and advancing integrative brain-body research.
19 April 2024 in IEEE Transactions on Radiation and Plasma Medical Sciences
Early and accurate identification of intracranial hemorrhage (ICH) is crucial for treatment, but the inherently low contrast resolution of CT imaging poses challenges in identification of specific cerebral regions, impacting effective and timely clinical decision-making. We propose BraSEDA, a CT-based unsupervised domain adaptation (UDA) model designed to assist in the identification of brain regions. BraSEDA framework utilizes a cross-modal instance normalization (CMIN) module for enhancing CT image structural features and creating high-quality pseudo MR images. A multi-level CMIN architecture is also introduced for further improvement. The BraSEDA framework improved the quality of pseudo MR images in head CT to MR domain adaptation task, as reflected by the lowest FID scores 95.0± 12.1 (p-value<0.001) with and highest BC scores 0.915± 0.396 (p-value <0.01), compared to other UDA methods. It also demonstrated superior segmentation performance in other four traditional medical UDA tasks. The BraSEDA framework effectively addresses the challenge of low-contrast areas in CT brain images for ICH cases. It enables precise identification of low-contrast brain regions, making it a potential tool for assisting in the accurate and prompt initial diagnosis of ICH in emergency medical settings. The code and well-trained model has been publicly available: https://github.com/YixinChen-AI/BraSEDA.