Transrectal ultrasonography (TRUS) has been widely accepted as a popular imaging modality for evaluating the lower rectum, anal sphincters, and pelvic floor in patients with various anorectal diseases. It provides excellent visualization of the layers of the rectal wall and of the anatomy of the anal canal. TRUS is an accurate tool for the staging of primary rectal cancer, especially for early stages. Although magnetic resonance imaging is a modality complementary to TRUS with advantages for evaluating the mesorectum, external sphincter, and deep pelvic inflammation, three-dimensional ultrasonography improves the detection and characterization of perianal fistulas and therefore plays a crucial role in optimal treatment planning. The operator should be familiar with the anatomy of the rectum and pelvic structures relevant to the preoperative evaluation of rectal cancer and other anal canal diseases, and should have technical proficiency in the use of TRUS combined with an awareness of its limitations compared to magnetic resonance imaging.
We aimed to compare the capabilities of two leading large language models (LLMs), GPT-4 and Gemini, in analyzing serial radiology reports, to highlight oncological issues that require further clinical attention. This study included 205 patients, each with two consecutive radiological reports. We designed a prompt comprising a three-step task to analyze report findings using LLMs. To establish a ground truth, two radiologists reached a consensus on a six-level categorization, comprising tumor findings (categorized as improved, stable, or aggravated), "benign", "no tumor description," and "other malignancy." The performance of GPT-4 and Gemini was then compared based on their ability to match corresponding findings between two radiological reports and accurately reflect these categories. In terms of accuracy in matching findings between serial reports, the proportion of correctly matched findings was significantly higher for GPT-4 (96.2%) than for Gemini (91.7%) (P < 0.01). For oncological issue identification, the precision for tumor-related finding determinations, recall, and F1-scores were 0.68 and 0.63 (P = 0.006), 0.91 and 0.80 (P < 0.001), and 0.78 and 0.70 for GPT-4 and Gemini, respectively. GPT-4 was more accurate than Gemini in determining the correct tumor status for tumor-related findings (P < 0.001). This study demonstrated the potential of LLM-assisted analysis of serial radiology reports in enhancing oncological surveillance, using a carefully engineered prompt. GPT-4 showed superior performance compared to Gemini in matching corresponding findings, identifying tumor-related findings, and accurately determining tumor status.
20 July 2022 in Computer Methods and Programs in Biomedicine