Radiology reports can change during workflows, especially when residents draft preliminary versions that attending physicians finalize. We explored how large language models (LLMs) and embedding techniques can categorize these changes into textual, semantic, or clinically actionable types. We evaluated 400 adult CT reports drafted by residents against finalized versions by attending physicians. Changes were rated on a five-point scale from no changes to critical ones. We examined open-source LLMs alongside traditional metrics like normalized word differences, Levenshtein and Jaccard similarity, and text embedding similarity. Model performance was assessed using quadratic weighted Cohen's kappa (κ), (balanced) accuracy, F₁, precision, and recall. Inter-rater reliability among evaluators was excellent (κ = 0.990). Of the reports analyzed, 1.3 % contained critical changes. The tested methods showed significant performance differences (P < 0.001). The Qwen3-235B-A22B model using a zero-shot prompt, most closely aligned with human assessments of changes in clinical reports, achieving a κ of 0.822 (SD 0.031). The best conventional metric, word difference, had a κ of 0.732 (SD 0.048), the difference between the two showed statistical significance in unadjusted post-hoc tests (P = 0.038) but lost significance after adjusting for multiple testing (P = 0.064). Embedding models underperformed compared to LLMs and classical methods, showing statistical significance in most cases. Large language models like Qwen3-235B-A22B demonstrated moderate to strong alignment with expert evaluations of the clinical significance of changes in radiology reports. LLMs outperformed embedding methods and traditional string and word approaches, achieving statistical significance in most instances. This demonstrates their potential as tools to support peer review.

Background: Radiology trainees require efficient, accurate, and accessible resources to master complex imaging techniques and identify findings that guide clinical decision-making. Large language models (LLMs) are emerging as promising tools for medical education and clinical workflows, offering the potential to enhance learning by providing instant feedback, aiding in diagnostic accuracy, and offering personalized learning experiences. However, systematic comparisons of LLMs for radiology education and clinical support remain limited, particularly regarding differences across subspecialties and resident experience levels. Objective: This study aimed to evaluate and compare the response quality of 2 state-of-the-art reasoning-based LLMs, namely DeepSeek-R1 and ChatGPT-o1, as clinical and radiology residency support tools, comparing performance across clinical and didactic dimensions, including text- and image-based responses. Methods: Overall, 27 radiology questions covering 9 radiological subspecialties were answered by both LLMs. Additionally, 6 image-based questions were presented only to ChatGPT-o1 due to its image processing capabilities. Responses were independently rated by 7 radiology residents (postgraduate years 2‐5) across 9 rating criteria grouped into 3 dimensions (factual accuracy, clinical practicality, and didactic value), using a 5-point Likert scale. Statistics compared LLMs, reader experience, and response types for text- as well as image-based for ChatGPT-o1 queries. Results: DeepSeek-R1 consistently outperformed ChatGPT-o1 across all rating dimensions, with highly significant differences across all criteria (mean ratings: DeepSeek-R1 4.51, SD 0.73 vs ChatGPT-o1 3.73, SD 0.98; P <.001). In an exploratory subspecialty-level analysis, DeepSeek-R1 descriptively outperformed ChatGPT-o1 across all subspecialties. For both LLMs accumulated, junior residents tended to rate slightly higher than seniors in 7 of 9 criteria, although differences were not statistically significant. However, for ChatGPT-o1, junior residents rated significantly higher in overall score across all criteria (juniors 3.81, SD 0.64 vs seniors 3.63, SD 0.65; P =.02). Image-based responses by ChatGPT-o1 scored significantly lower than text-based (mean 3.19, SD 1.42; P =.007), particularly in factual accuracy (mean 2.75, SD 1.45; P <.001) and clinical practicality (mean 3.11, SD 1.47; P =.03). Conclusions: Both DeepSeek-R1 and ChatGPT-o1 demonstrate promising potential on simulated radiology question sets designed for educational and clinical contexts, with DeepSeek-R1 outperforming ChatGPT-o1 across all evaluated criteria. These results emphasize the value of open-source models for educational use and provide early evidence that LLMs may support radiology resident training under controlled conditions; however, their real-world educational and clinical effects require further investigation. Future research should prospectively evaluate how LLMs can be integrated into radiology training, assess their impact alongside conventional teaching methods, and investigate multimodal capabilities to better reflect realistic clinical scenarios.

Background: Tracking differential growth of secondary liver metastases is important for early detection of progression but remains challenging due to variable tumor growth rates. We aimed to automate accurate, consistent, and efficient longitudinal monitoring. Methods: We developed an automatic 3D segmentation and tracking algorithm to quantify differential growth, tested on contrast-enhanced MRI follow-ups of patients with neuroendocrine liver metastases (NELMs). The output was integrated into a decision support tool to distinguish between progressive disease, stable disease, and partial/complete response. A user study involving an expert group of seven expert radiologists evaluated its impact. Group comparisons used the Friedman test with post hoc analyses. Results: Our algorithm detected 991 metastases in 30 patients: 13% new, 30% progressive, 18% stable, and 18% regressive; the remainder were either too small to measure (15%) or merged with another metastasis in the follow-up assessment (6%). Diagnostic accuracy improved with additional information on hepatic tumor load and differential growth, albeit not significantly (p = 0.72). The diagnosis time increased (p < 0.001). All radiologists found the method useful and expressed a desire to integrate it in existing diagnostic tools. Conclusions: We automated segmentation and quantification of individual NELMs, enabling comprehensive longitudinal analysis of differential tumor growth with the potential to enhance clinical decision-making.