12 April 2022 in IEEE Journal of Biomedical and Health Informatics
Automated layer segmentation plays an important role for retinal disease diagnosis in optical coherence tomography (OCT) images. However, the severe retinal diseases result in the performance degeneration of automated layer segmentation approaches. In this paper, we present a robust semi-supervised layer segmentation network to relieve the model failures on abnormal retinas. We obtain the lesion features from the labeled images with disease-balanced distribution, and utilize the unlabeled images to supplement the layer structure information. Specifically, in our method, the cross-consistency training is utilized over the predictions of different decoders, and we enforce a consistency between different decoder predictions to improve the encoder's representation. Then, we propose a sequence prediction branch based on self-supervised manner, which is designed to predict the position of each jigsaw puzzle to obtain sensory perception of the retinal layer structure. To this task, a layer spatial pyramid pooling (LSPP) module is designed to extract multi-scale layer spatial features. Furthermore, we use the optical coherence tomography angiography (OCTA) to supplement the information damaged by diseases. The experimental results illustrate that our method achieves more robust results compared with current supervised segmentation methods. Meanwhile, advanced segmentation performance can be obtained compared with state-of-the-art semi-supervised segmentation methods.
26 December 2023 in Medical Image Analysis
1 December 2012 in Computers & Education
7 December 2025 in Medical Image Analysis
3 October 2023 in IEEE Transactions on Emerging Topics in Computational Intelligence
Optical coherence tomography angiography (OCTA) is regarded as the best visualization of retinal vasculature and is widely used. However, the acquisition of OCTA is still limited by the device. In this study, we aim to generate OCTA from optical coherence tomography (OCT) images for devices without the OCTA imaging module. We proposed a novel unsupervised 3D domain adaption approach, termed Patch-based 3D Multi-scale Feature Adaption Networks (PMFAN), to transfer the OCTA generator trained on one device to other devices. To perform domain adaption, we first map the OCT of different devices to shared-feature spaces. To maintain fine-grained vessel features, we encode the OCT into a set of multi-scale feature codes. Besides, we model the domain adaption in 3D with a patch-based data manner. To reduce inaccurate feature coding due to incomplete context at the border of patches, we propose a novel context-enhanced encoder. Then, we generate OCTA from the feature codes by a domain-invariant OCTA generator. The experimental results on a dataset from two devices demonstrate that our method outperforms other unsupervised domain adaptation methods and image-translation-based methods in terms of the cross-device OCT to OCTA translation.
8 June 2026 in Nature Communications
The non-invasive assessment of renal masses remains a critical challenge in urologic oncology, where diagnostic uncertainty frequently causes overtreatment. Here, we develop RenalCLIP, a vision-language foundation model for precision oncology in kidney cancer. Utilizing 27,866 computed tomography scans from 8809 patients across diverse multi-center cohorts, we employ a two-stage pre-training strategy to align domain-specific visual and textual representations. RenalCLIP achieves enhanced performance and generalizability across ten core clinical tasks, spanning anatomical assessment, diagnostic classification, and survival prediction, significantly outperforming state-of-the-art general-purpose foundation models. Furthermore, RenalCLIP demonstrates strong data efficiency in diagnostic classification, achieving peak baseline performance using only 20% of the training data. The model also exhibits robust zero-shot diagnostic capabilities, effective image-text retrieval, and high-quality medical report generation. Our findings establish RenalCLIP as a powerful, generalizable tool to enhance diagnostic precision, refine prognostic stratification, and personalize the management of renal masses.