15 July 2025 in Computers in Biology and Medicine
Content-based image retrieval (CBIR) has the potential to significantly improve diagnostic aid and medical research in radiology. However, current CBIR systems face limitations due to their specialization to certain pathologies, limiting their utility. On the other hand, several vision foundation models have been shown to produce general-purpose visual features. Therefore, in this work, we propose using vision foundation models as powerful and versatile off-the-shelf feature extractors for content-based image retrieval. Our contributions include: (1) benchmarking a diverse set of vision foundation models on an extensive dataset comprising 1.6 million 2D radiological images across four modalities and 161 pathologies; (2) identifying weakly-supervised models, particularly BiomedCLIP, as highly effective, achieving a P@1 of up to 0.594 (P@3: 0.590, P@5: 0.588, P@10: 0.583), comparable to specialized CBIR systems but without additional training; (3) conducting an in-depth analysis of the impact of index size on retrieval performance; (4) evaluating the quality of embedding spaces generated by different models; and (5) investigating specific challenges associated with retrieving anatomical versus pathological structures. Despite these challenges, our research underscores the vast potential of foundation models for CBIR in radiology, proposing a shift towards versatile, general-purpose medical image retrieval systems that do not require specific tuning. Our code, dataset splits and embeddings are publicly available here.
25 October 2024 in Journal of the American Medical Informatics Association
Objective: Federated Learning (FL) enables collaborative model training while keeping data locally. Currently, most FL studies in radiology are conducted in simulated environments due to numerous hurdles impeding its translation into practice. The few existing real-world FL initiatives rarely communicate specific measures taken to overcome these hurdles. To bridge this significant knowledge gap, we propose a comprehensive guide for real-world FL in radiology. Minding efforts to implement real-world FL, there is a lack of comprehensive assessments comparing FL to less complex alternatives in challenging real-world settings, which we address through extensive benchmarking. Materials and Methods: We developed our own FL infrastructure within the German Radiological Cooperative Network (RACOON) and demonstrated its functionality by training FL models on lung pathology segmentation tasks across six university hospitals. Insights gained while establishing our FL initiative and running the extensive benchmark experiments were compiled and categorized into the guide. Results: The proposed guide outlines essential steps, identified hurdles, and implemented solutions for establishing successful FL initiatives conducting real-world experiments. Our experimental results prove the practical relevance of our guide and show that FL outperforms less complex alternatives in all evaluation scenarios. Discussion and Conclusion: Our findings justify the efforts required to translate FL into real-world applications by demonstrating advantageous performance over alternative approaches. Additionally, they emphasize the importance of strategic organization, robust management of distributed data and infrastructure in real-world settings. With the proposed guide, we are aiming to aid future FL researchers in circumventing pitfalls and accelerating translation of FL into radiological applications.
24 June 2026 in Radiology: Artificial Intelligence
Purpose To develop and systematically evaluate an iterative training approach, termed the expert-guided annotation loop, for efficient reference standard medical image segmentation, including assessment of two sample-selection strategies and real-world clinical implementation. Materials and Methods This retrospective study included ten datasets comprising 1948 CT or MRI examinations from autosomal dominant polycystic kidney disease, prostate cancer, uveal melanoma, thyroid eye disease, and non-small cell lung cancer patients. nnU-Net segmentation models were iteratively trained using an expert-guided annotation loop with random or active learning-based sample selection. In each iteration, additional samples were added to the training set, and model-generated presegmentations were corrected by expert radiologists to create reference standard annotations. Expert time required for manual segmentation versus presegmentations correction was measured. Model performance and efficiency were assessed using nonparametric tests, and cost savings were estimated for kidney and tumor segmentation using probabilistic sensitivity analysis. Feasibility of end-to-end no-code implementation was evaluated. Results Fifty-seven segmentation models were trained and evaluated. Final model Dice scores ranged from 0.67–0.97 for organ segmentation and from 0.64–0.69 for lung tumor segmentation across internal and external test sets. Maximum expert time savings were 90.3% for kidney and 48.2% for tumor segmentation ( P < .001 and P = .003), corresponding to estimated per-examination cost savings of $14.30[95% CI: $5.94, $26.87] and $5.63[95% CI: $-7.26, $26.09], respectively. No-code execution of the expert-guided annotation loop was feasible. Conclusion The expert-guided annotation loop reduced expert annotation time and enabled estimated cost savings while producing high-quality reference standard CT and MRI segmentations. The no-code workflow was implemented in a clinical environment. © RSNA, 2026