Background: Recent studies, including those by the National Board of Medical Examiners, have highlighted the remarkable capabilities of recent large language models (LLMs) such as ChatGPT in passing the United States Medical Licensing Examination (USMLE). However, there is a gap in detailed analysis of LLM performance in specific medical content areas, thus limiting an assessment of their potential utility in medical education. Objective: This study aimed to assess and compare the accuracy of successive ChatGPT versions (GPT-3.5, GPT-4, and GPT-4 Omni) in USMLE disciplines, clinical clerkships, and the clinical skills of diagnostics and management. Methods: This study used 750 clinical vignette-based multiple-choice questions to characterize the performance of successive ChatGPT versions (ChatGPT 3.5 [GPT-3.5], ChatGPT 4 [GPT-4], and ChatGPT 4 Omni [GPT-4o]) across USMLE disciplines, clinical clerkships, and in clinical skills (diagnostics and management). Accuracy was assessed using a standardized protocol, with statistical analyses conducted to compare the models’ performances. Results: GPT-4o achieved the highest accuracy across 750 multiple-choice questions at 90.4%, outperforming GPT-4 and GPT-3.5, which scored 81.1% and 60.0%, respectively. GPT-4o’s highest performances were in social sciences (95.5%), behavioral and neuroscience (94.2%), and pharmacology (93.2%). In clinical skills, GPT-4o’s diagnostic accuracy was 92.7% and management accuracy was 88.8%, significantly higher than its predecessors. Notably, both GPT-4o and GPT-4 significantly outperformed the medical student average accuracy of 59.3% (95% CI 58.3‐60.3). Conclusions: GPT-4o’s performance in USMLE disciplines, clinical clerkships, and clinical skills indicates substantial improvements over its predecessors, suggesting significant potential for the use of this technology as an educational aid for medical students. These findings underscore the need for careful consideration when integrating LLMs into medical education, emphasizing the importance of structured curricula to guide their appropriate use and the need for ongoing critical analyses to ensure their reliability and effectiveness.

Objective: To develop and evaluate the effectiveness of domain‐specific customization in large language models (LLMs) by assessing the performance of the ENT GPT Assistant (E‐GPT‐A), a model specifically tailored for otolaryngology.Study Design: Comparative analysis using multiple‐choice questions (MCQs) from established otolaryngology resources.Setting: Tertiary care academic hospital.Methods: Two hundred forty clinical‐vignette style MCQs were sourced from BoardVitals Otolaryngology and OTOQuest, covering a range of otolaryngology subspecialties (n = 40 for each). The E‐GPT‐A was developed using targeted instructions and customized to otolaryngology. The performance of E‐GPT‐A was compared against top‐performing and widely used artificial intelligence (AI) LLMs, including GPT‐3.5, GPT‐4, Claude 2.0, and Claude 2.1. Accuracy was assessed across subspecialties, varying question difficulty tiers, and in diagnostics and management.Results: E‐GPT‐A achieved an overall accuracy of 74.6%, outperforming GPT‐3.5 (60.4%), Claude 2.0 (61.7%), Claude 2.1 (60.8%), and GPT‐4 (68.3%). The model performed best in allergy and rhinology (85.0%) and laryngology (82.5%), whereas showing lower accuracy in pediatrics (62.5%) and facial plastics/reconstructive surgery (67.5%). Accuracy also declined as question difficulty increased. The average correct response percentage among otolaryngologists and otolaryngology trainees was 71.1% in the question set.Conclusion: This pilot study using the E‐GPT‐A demonstrates the potential benefits of domain‐specific customizations of language models for otolaryngology. However, further development, continuous updates, and continued real‐world validation are needed to fully assess the capabilities of LLMs in otolaryngology.

Acute invasive fungal sinusitis (AIFS) is a rapidly progressing, life-threatening infection. The advent of immunomodulatory therapies has expanded the population susceptible to AIFS. In this case report, we describe a patient who defies the conventional profile of AIFS. This 70-year-old woman is immunocompetent and non-diabetic, with a history of multiple sclerosis (MS) and ongoing treatment with glatiramer acetate (GA), immunomodulator. She came to the emergency room due to acute vision changes, and images revealed an enhancing mass in the left pterygopalatine fossa. The diagnosis of AIFS was confirmed by biopsy in the operating room. Subsequently, anti-fungal therapies were initiated with a follow-up operative debridement. Follow-up magnetic resonance imaging (10 months since treatment) showed no progression of AIFS. GA was discontinued since the diagnosis, and MS has remained stable. Recognizing this unique group of atrisk patients is crucial, as early detection and treatment play a pivotal role in preventing significant morbidity and mortality.

Purpose: To evaluate a treatment-naïve cohort of patients with proliferative diabetic retinopathy (PDR) and assess the costs of panretinal photocoagulation (PRP) initially performed in the operating room or the clinic, incorporating the cost of the additional follow-up procedures required for each treatment group. Methods: A retrospective review was performed of patients with PDR initially treated with PRP in the operating room or the clinic. Cost data were derived from Current Procedural Terminology codes, and estimated mean facility costs were provided. For each cohort, negative binomial regressions were used to compare counts of subsequent interventions, and visual acuity (VA) outcomes and dollars per patient-treatment year were compared with paired t tests. Results: Eighty-two eyes of 53 patients met the inclusion criteria. The operating room cohort included 56 eyes of 38 patients, while the clinic cohort included 26 eyes of 16 patients. Patients had a minimum follow-up of more than 3 years. The operating room cohort required fewer subsequent PRP treatments (mean, 1.0 vs 2.1; P < .05) and surgeries (mean, 0.3 vs 0.7; P < .05) than the clinic cohort. The mean best-corrected VA (BCVA) after treatment was significantly better in the operating room cohort (0.30 ± 0.40 logMAR; Snellen equivalent, 20/39.9) than the clinic cohort (0.75 ± 0.81 logMAR; Snellen equivalent, 20/112.5) ( P < .05). The cost per patient-treatment year was similar between the cohorts (operating room, $5,886.79; clinic, $5,657.50) ( P = .75). Conclusions: PRP initially administered in the operating room was equal in cost to clinic administration and required fewer subsequent PRP sessions and surgical treatments. In addition, there was a significant improvement in the final BCVA.