Foundation models, including large language models and vision-language models (VLMs), have revolutionized artificial intelligence by enabling efficient, scalable, and multimodal learning across diverse applications. By leveraging advancements in self-supervised and semi-supervised learning, these models integrate computer vision and natural language processing to address complex tasks, such as disease classification, segmentation, cross-modal retrieval, and automated report generation. Their ability to pretrain on vast, uncurated datasets minimizes reliance on annotated data while improving generalization and adaptability for a wide range of downstream tasks. In the medical domain, foundation models address critical challenges by combining the information from various medical imaging modalities with textual data from radiology reports and clinical notes. This integration has enabled the development of tools that streamline diagnostic workflows, enhance accuracy (ACC), and enable robust decision-making. This review provides a systematic examination of the recent advancements in medical VLMs from 2022 to 2024, focusing on modality-specific approaches and tailored applications in medical imaging. The key contributions include the creation of a structured taxonomy to categorize existing models, an in-depth analysis of datasets essential for training and evaluation, and a review of practical applications. This review also addresses ongoing challenges and proposes future directions for enhancing the accessibility and impact of foundation models in healthcare.

Lee S, et al. Yonsei Med J. 2022 Jan;63(Suppl):S93-S107. https://doi.org/10.3349/ymj.2022.63.S93

Recently, several efforts have been made to develop the deep learning (DL) algorithms for automatic detection and segmentation of brain metastases (BM). In this study, we developed an advanced DL model to BM detection and segmentation, especially for small-volume BM. From the institutional cancer registry, contrast-enhanced magnetic resonance images of 65 patients and 603 BM were collected to train and evaluate our DL model. Of the 65 patients, 12 patients with 58 BM were assigned to test-set for performance evaluation. Ground-truth for BM was assigned to one radiation oncologist to manually delineate BM and another one to cross-check. Unlike other previous studies, our study dealt with relatively small BM, so the area occupied by the BM in the high-resolution images were small. Our study applied training techniques such as the overlapping patch technique and 2.5-dimensional (2.5D) training to the well-known U-Net architecture to learn better in smaller BM. As a DL architecture, 2D U-Net was utilized by 2.5D training. For better efficacy and accuracy of a two-dimensional U-Net, we applied effective preprocessing include 2.5D overlapping patch technique. The sensitivity and average false positive rate were measured as detection performance, and their values were 97% and 1.25 per patient, respectively. The dice coefficient with dilation and 95% Hausdorff distance were measured as segmentation performance, and their values were 75% and 2.057 mm, respectively. Our DL model can detect and segment BM with small volume with good performance. Our model provides considerable benefit for clinicians with automatic detection and segmentation of BM for stereotactic ablative radiotherapy.

Benign paroxysmal positional vertigo (BPPV), the most common vestibular disorder, is diagnosed by an examiner changing the posture of the examinee and inducing nystagmus. Among the diagnostic methods used to observe nystagmus, video-nystagmography has been widely used recently because it is non-invasive. A specialist with professional knowledge and training in vertigo diagnosis is needed to diagnose BPPV accurately, but the ratio of vertigo patients to specialists is too high, thus necessitating the need for automated diagnosis of BPPV. In this paper, a convolutional neural network-based nystagmus extraction system, ANyEye, optimized for video-nystagmography data is proposed. A pupil was segmented to track the exact pupil trajectory from real-world data obtained during field inspection. A deep convolutional neural network model was trained with the new video-nystagmography dataset for the pupil segmentation task, and a compensation algorithm was designed to correct pupil position. In addition, a slippage detection algorithm based on moving averages was designed to eliminate the motion artifacts induced by goggle slippage. ANyEye outperformed other eye-tracking methods including learning and non-learning-based algorithms with five-pixel error detection rate of 91.26%.